In less than a week the Mac package for SOFA Statistics has been downloaded over 100 times, representing nearly a quarter of all downloads of version 0.9.10.
Example output on OS X
A positive response was expected given recent survey results at the main SOFA Statistics website – 35% of respondents wanted the Mac package as the next thing to add to the project. And about 14% of visitors to the main website are using a Mac OS. But it was still nice to see the response.
SOFA Statistics has been packaged for Mac OS X (Leopard and Snow Leopard). It only seems right that an application emphasising ease-of-use and beautiful output should be available for Apple users.
SOFA Statistics on OS X
The installer is supplied as a mpkg (metapackage) file inside a disk image. Open the mpkg to install.
SOFA mac installer
The same application now works much the same on Windows XP, Vista, and Windows 7, Ubuntu/Linux Mint, and Mac OS X Leopard and Snow Leopard.
SOFA controls on mac
Even if you are not a Mac user, there are a few extras in the new version:
There are also a few bug fixes:
You can download version 0.9.10 now: SOFA downloads
NB 0.9.10 (zero point nine point ten) is newer than 0.9.9.
SOFA Statistics has now reached the 10,000 download milestone on Sourceforge. And the rate of downloading has been steadily increasing. In May 2009 there were only 30 downloads whereas in May 2010 there are likely to be over 50 times that. The main area of work at the moment is creating a Mac package. Follow this blog or the twitter feed (http://twitter.com/sofastatistics) to stay informed.
10000 downloads!
SOFA Statistics is growing in popularity, if SourceForge downloads are any indication.
Sourceforge Download Trend
Of course it is early days yet for the project, with version 1.0 not expected until later this year, but current trends are encouraging. It seems there is a gap in the market for an open source statistics, analysis, and reporting package aimed at non-specialist statisticians.
R is an open source programming language and software environment for statistics. And it is not just any old programming language – it is the dominant system for open source statistics. So was it fair to call R an “epic fail” as Dr. AnnMaria De Mars did in her notorious blog post The Next Big Thing?
Clearly R has been a massive success and it has a vibrant and lively community, many of whom were galvanised into making a response by the Epic Fail blog (see An article attacking R gets responses from the R blogosphere – some reflections on the phenomenon and R and the Next Big Thing as an example). So on what terms could it be considered a failure? For De Mars, successful software will be usable by the vast majority of people – not just programmers and others comfortable with command line interfaces.
… if you even LOOK at R code – bug-free or not, compilable or not – it should be evident that this is not how the average person uses a computer. If we are talking about something that is going to be used by a large number of people, R is not it (Comment by De Mars on her own blog post – The Next Big Thing).
… If your target market is “People who own cars that drive from point A to point B” that is much BIGGER than “people who work on engines”. If you are looking for a job making things or selling things or providing services, the former is more likely to pay off for you than the latter.
Telling people that if they can’t appreciate an internal combustion engine they are too stupid to own a car probably won’t help, either.” (The Next Big Thing).
And in these terms, De Mars has a point. For many users, R needs a GUI. I like this quote tweeted by ravkalia (a big fan of R BTW): “Overheard at a computing meeting: ‘R is not a programming language, it’s a statistics package with the GUI missing.'” Of course there are various projects to provide a GUI interface for R but it can be argued there are limits to how far that can go given the inherent flexibility of R as an environment. Yihui Xie recently commented – “I prefer the command-line due to its flexibility. GUI cannot hold infinite components (buttons, drop-lists, check-boxes, …), whereas there are almost infinite possibilities in commands.” (r-is-an-epic-fail).
On her other points regarding R and data visualisation, and analysis of enormous quantities of unstructured data, De Mars is on shakier ground, but the observations about the mainstream preference for looking and clicking are valid.
So how does this relate to SOFA Statistics? SOFA stands for Statistics Open For All, which gives a strong hint as to where SOFA is aiming in terms of user interfaces and target audience. In practice this means:
Interface chaos
Some commentators have implied that a GUI is not important because the sorts of people who do statistics will also be comfortable with basic programming. But this is not always true. And lots more people, by several orders of magnitude, need to run basic statistical analyses than just specialist statisticians. Karen Grace-Martin put it especially well in her response to the Epic Fail post:
“I primarily help researchers, mainly in biology and social science, apply statistics to their research. They are not doing “business analytics,” do not have enormous databases, and really have no need to program anything beyond what SAS or SPSS syntax does. They are not programmers or statisticians, and they don’t have backgrounds in programming or math.
I believe they are the kinds of users of statistics that you are referring to and I agree with you wholeheartedly that they are probably the majority of statistics users and they have no need for a programming language. They don’t want to nor need to program new statistical procedures.
There are clearly people who do, but I agree they’re not the majority. At least not in the fields I work.” (The Next Big Thing).
Even full-time specialist statisticians may find it easier to use a simple GUI for basic data exploration e.g. generating simple frequency tables and cross tabs. It has been suggested that people should expect to use more than one package (SPSS, SAS, R, Stata, JMP? Choosing a Statistical Software Package or Two) SOFA Statistics may be a useful complement to R for many users.
And ease of use should not be premised on the assumption that people will be heavy users of the package – or of statistics in general, for that matter. The program needs to make it easy to become productive in a hurry.
Lucid spirals demo
Even the program itself needs to look good:
Form for selecting appropriate statistical test
* In the Zen of Python (type import this into your Python interpreter) there is this gem: “There should be one– and preferably only one –obvious way to do it.”
Here is an error message that I get a lot from a popular R package.
‘Error in optim(0, f, control = control, hessian = TRUE, method = “BFGS”) :
non-finite finite-difference value [1]’
I know what that means. Would an analyst?
(Jon Peck – in response to The Next Big Thing)
But one thing is clear to me: R aims at people who know what they are doing. Absolutely. You can see this with standard output in R which is very minimalistic. You must ASK R what you want from it. SAS and SPSS put everything out. And therefore you need to know how to program in R to use it, really. But if you do, you feel bound and limited with SAS or SPSS. (comment by mocianmomo in response to SAS v. R: Ease of learning).
Example SOFA script in Python
Although this latest release has many, many enhancements and fixes (full details below), the most pleasing change for most users will probably be the availability of some attractive new output styles. It is also easier to select and apply them.
The 3 new styles are:
Grey spirals demo
Lucid spirals demo
Pebbles demo
Output styles can be selected using a simple drop-down list.
Style selection
The full list of new features is:
There have also been numerous bug fixes:
The latest set of changes are the direct result of user input. A user emailed us to say he found working with the output reports a bit confusing and the end result was a better way of viewing and organising output reports.
So here’s how it works. If you start saving output, it goes into the default html report file. If you want it to go into an existing html file, you select that using Browse and it will be added there. And if you want to create a new output file, you can just browse to the correct folder and enter the file name. Pretty simple and flexible as well.
Make new output report
As for viewing output reports, it has always been possible to use your file manager to locate your output file and double click it (thus opening it as a tab in your default browser e.g. Firefox). Now it is much easier – just click on the new View button next to the report name and it will automatically open as a tab in your default browser.
New output view button
Your output opened in your browser
A few bugs were fixed as well and there was a major set of changes to prevent future bugs related to database connections:
Whether SOFA Statistics finally settles on RaphaelJS …
RaphaelJS Pie Chart (NB dynamic in live version)
… or Dojo for output charting I am glad that it will be using SVG (and Javascript). These technologies, and support from them by mainstream web browsers, is only going to get faster and better.
Although the example below is not a graph it gives a taste of what is possible using these technologies: http://svg-wow.org/audio/animated-lyrics.html. Imagine being able to add comments and highlights directly to a chart which you can share with anyone. No proprietary viewer necessary :-).
BTW the reason output charting is not yet available is because I’m waiting till the wxWebKit widget supports it. wxWebKit is the technology SOFA Statistics uses to display HTML (web content) internally. The good news is that the improvements to wxWebKit should be ready by June or July.
There are four big changes in the latest version. They’re not really new features as such – the changes are mainly focused on making SOFA Statistics more pleasant to use. We hope you like it :-).
1) When designing or redesigning data tables, users now have visual feedback on the changes made in the form of a demonstration table:
Data design form
2) The Projects form has been made a bit more attractive and coherent. Hopefully, it will be easier to use:
Project form refreshed
3) Demonstration report tables are more visually distinct from actual report tables (i.e. those run off the data):
Demo tables more faint
4) Better guidance when asking if a csv file or spreadsheet has a header or not:
Import has header
Here is the full list of improvements:
And there are bug fixes:
The latest version of SOFA Statistics (0.9.6) is well worth the upgrade. Deep down, I was never happy with the approach to creating frequency tables. It was elegant in some ways to think of frequency tables as a special case of a rows x columns report tables but it was still a mistake. Now there are four types of report table – Frequencies, Crosstabs, Row summaries (means etc), and simple Data Lists. Here is how to make a simple Frequency Table for Age Groups with column percentages:
New Frequency Tables Interface
Also, large data tables load almost instantly now. The table I just opened was over 200,000 rows and it simply appeared as soon as I clicked on the Open button.
The third major advance is for people who want to work with spreadsheets. Until now, Excel has been the only option, which is only available on Windows systems. Now users can enter data into an OpenOffice Calc or Gnumeric spreadsheet and it should be possible to import it successfully. Please let me know how that goes.
ODS spreadsheets
Here is the full list of new features:
The bug fixes will probably be as important, especially if you have experienced any of them: