Statistics Open For All

The features page of SOFA Statistics (http://www.sofastatistics.com/features.php) has a number of bullet points listing features. There is nothing wrong with that. But bullet points don’t tell you how well a feature is implemented or how well it integrates with the rest of the program. Here is an interesting wikipedia entry of relevance: http://en.wikipedia.org/wiki/Bullet-point_engineering . The goal for SOFA Statistics will be to limit the range of features (in the core) but to increase the depth and quality of them. E.g. greater support for users, help with decision-making on appropriate tests, greater ease sharing output or automating report making, more languages supported etc. Beyond the core, it is hoped there will be numerous modules available. But the emphasis for now is on the value of the features added rather than the sheer number of them.

The latest release has many of the features I have been wanting to add for a long time. In particular, there are more support charts to help users decide if a particular analysis is appropriate. If you run an ANOVA, for example, how normal are all the distributions for the subgroups? And what about variability? SOFA Statistics provides visual assistance for deciding (e.g. histograms with superimposed normal distribution curves) alongside the numerical results of appropriate tests (e.g. the O’Brien homogeneity of variance test). Instead of expecting the user to know the separate steps they ought to take, SOFA Statistics bundles them up together and tries to provide guidance and interpretation. Which is arguably how it should be. For example, how can a user properly interpret an R result from a correlation without a scatterplot? Many users won’t have studied statistics formally for a long time (if at all) and it is easy to be uncertain about exactly what all the rules are.

Re: portable output, SOFA Statistics reports are designed for viewing in web browsers (e.g. Firefox). Now that these reports include images it has become important to make sure they are easily portable. To that end, all internal links to images are relative. This means you can copy a report and the subfolder of its images (sharing the name of the report) anywhere and have the report work properly. It has never been easier to share the results of your analyses.

Here is a full list of the changes:

• ANOVA output now includes histograms for each sample with superimposed normal distribution curves. It also shows kurtosis, skew, and an omnibus measure of normality for each sample as well as the O’Brien homogeneity of variance test. Explanatory footnotes have been added to the output.
  
  

  Histograms for subgroups of ANOVA

• Spearman’s and Pearson’s correlation output now includes scatterplots and lines of best fit.
  
  

  Scatterplot for assessing linear correlation

• All html reports are portable along with their images (stored in a subfolder of the same name).
• When titles/subtitles are being changed, the rest of the example report table stays the same. This removes an annoying “flicker” effect when typing in titles/subtitles.
• The redundant Clear button has been removed from Statistical Test dialogs.
• An hourglass displays when opening statistics tests and report tables for the first time in case of a brief delay on first use.

There have also been some important and edge-case bug fixes:

• All images are now uniquely named and stored in report-name-based subfolders if “added to report” has been selected, or in the internal folder otherwise. This guarantees the correct images will always be displayed and that saved HTML reports will work.
• The page break in independent t-test output has been repositioned to below the histograms.
• Changing to raw data display, and then changing table source, no longer prevents the example table from displaying.
• Internal footnotes in expanded output now work for Windows users.

0.9.1 is out and there have been a lot of improvements this time:

• All output now displays inside design dialogs. In the case of report tables, there is an option to expand output. This is especially important for displaying larger report tables on netbooks.
• Independent t-test output now includes two histograms with superimposed normal distribution curves.  It also shows kurtosis, skew, and an omnibus measure of normality for each sample as well as the O’Brien homogeneity of variance test. Explanatory footnotes have been added to the output e.g. explaining the p value or what kurtosis means.
  
  

  Independent t-test support graphics

• Guidance given on need to assess normality of each sample when more than one (part of test selection process).
• Hovering over cells in the data entry/editing grid displays appropriate value labels. E.g. hovering over 1 in a gender field may show the tooltip “Male” (if that label has been set up by the user)
• Can update variable details from within data editing/entry grid by right clicking on column labels. This ability is signalled by tooltips.
• Date format (e.g. US) is now automatically extracted from the operating system rather than requiring user preferences. Preferences now sets reporting explanation level (still not operational).

There have also been a few bug fixes:

• Fixed bug in independent t-test where the std dev displayed for sample b was actually that of sample a.
• Now copes with filter on a string variable when creating divider.
• Fixed loss-of-focus bug in Windows when typing titles and subtitles after having clicked html widget.

Version 0.8.15 has the following new features:

* Data filtering: If you’re only interested in data for one nation, for example, you can add a filter so reports are derived from that nation’s subset of the data. It is now easy to add, remove, and alter filters to individual tables. This feature makes analysis much more fluid. It is arguably one of the most important changes in long time. Check it out. Almost anywhere you can see tables listed, you can right click on the table name and set, alter, rename, or remove a filter.
* Support for US-style dates: SOFA Statistics now allows the setting of date entry format preferences, e.g. if you are in the US, Canada, the Philippines, Belize, Palau, or the Federated Republic of Micronesia, you may wish to allow mm/dd/yy as a date format for data entry instead of dd/mm/yy. Over time, there will no doubt be other preference options available.
* Default data table now has a datetime field to experiment with.
* More tooltips and misc usability improvements.

There have also been miscellaneous bug fixes including:

* Removed bug which meant you couldn’t immediately open a freshly-created table.

In addition there has been further expansion of test coverage and a fix of sorts for the Karmic launcher. There is a problem (hopefully temporary) where users of GTK applications can click a button, have it depress, and have nothing actually happen (unless they then click on the Enter key). What this means for SOFA Statistics is that when a Karmic Ubuntu user clicked on the new Preferences button, a preferences dialog wouldn’t appear. A partial solution has been to change the launcher slightly so it sets the GDK_NATIVE_WINDOWS environment variable. So instead of:

python /usr/share/pyshared/sofa/start.py

the launcher now invokes:
bash -c “GDK_NATIVE_WINDOWS=1 python /usr/share/pyshared/sofa/start.py”

… which solves the problem in many cases. If it doesn’t, the work-around is to mouse click on the Preferences button then press the Enter key. Fortunately, the Preferences button will hardly ever be used. Ideally, the problem will be fixed properly in newer versions of SOFA Statistics as the underlying code libraries are fixed.