SOFA Stats

SOFA Statistics continues to push on towards an eventual 1.0 release. Even though SOFA is still tagged as an early release on the main project website, and as a beta program on the SourceForge site, there have been over 7,000 downloads already. And the current release feels more solid to me than some of the expensive proprietary products I have used in my career as a researcher and analyst. Of course, please feel free to report any bugs that you find (http://groups.google.com/group/sofastatistics) so they can be fixed before the next release.

The basic list of new features is:

• SOFA now detects screen resolution and sizes height of dialogs/controls accordingly to better handle both netbooks and larger screens.
• Now able to delete rows in data entry/editing and configuration tables using the delete key.
• Added hyperlinks to main form for main project website and community website.
• The project configuration window can be completely resized making it easier to see all settings.
• The read-only default project form now has an OK button at the bottom right so it can be closed the same way as every other form.
• Added preliminary chart configuration form. Output charting still under development. In some ways this is the most important development, even though it doesn’t offer anything to the user in the current version apart from a taste of the likely interface.
• Hovering over image buttons now shows a different cursor in Ubuntu so users know that they are buttons they can click.

And the bug fixes of course:

• Spreadsheets lacking a header are now correctly imported including the first line.
• All forms of data import now cope with data which has altered field names in preparation for import into SQLite.
• Clustered bar charts in the Chi Square output work even if there are lots of bars missing from some clusters.
• Removed import statement that caused problems on some systems.
• Button clicks occurring while inside a configuration text box now work as expected on first attempt.
• If the primary key for an MS Access database lacks autonumbering, it will only be eligible for saving if it has a non-missing value.
• Hourglass doesn’t remain open too long when clicking Expand button in Ubuntu.

I’m currently running a small poll on the main project page about the most important things to work on next (http://www.sofastatistics.com). Please vote if you haven’t already.

The features page of SOFA Statistics (http://www.sofastatistics.com/features.php) has a number of bullet points listing features. There is nothing wrong with that. But bullet points don’t tell you how well a feature is implemented or how well it integrates with the rest of the program. Here is an interesting wikipedia entry of relevance: http://en.wikipedia.org/wiki/Bullet-point_engineering . The goal for SOFA Statistics will be to limit the range of features (in the core) but to increase the depth and quality of them. E.g. greater support for users, help with decision-making on appropriate tests, greater ease sharing output or automating report making, more languages supported etc. Beyond the core, it is hoped there will be numerous modules available. But the emphasis for now is on the value of the features added rather than the sheer number of them.

The latest release has many of the features I have been wanting to add for a long time. In particular, there are more support charts to help users decide if a particular analysis is appropriate. If you run an ANOVA, for example, how normal are all the distributions for the subgroups? And what about variability? SOFA Statistics provides visual assistance for deciding (e.g. histograms with superimposed normal distribution curves) alongside the numerical results of appropriate tests (e.g. the O’Brien homogeneity of variance test). Instead of expecting the user to know the separate steps they ought to take, SOFA Statistics bundles them up together and tries to provide guidance and interpretation. Which is arguably how it should be. For example, how can a user properly interpret an R result from a correlation without a scatterplot? Many users won’t have studied statistics formally for a long time (if at all) and it is easy to be uncertain about exactly what all the rules are.

Re: portable output, SOFA Statistics reports are designed for viewing in web browsers (e.g. Firefox). Now that these reports include images it has become important to make sure they are easily portable. To that end, all internal links to images are relative. This means you can copy a report and the subfolder of its images (sharing the name of the report) anywhere and have the report work properly. It has never been easier to share the results of your analyses.

Here is a full list of the changes:

• ANOVA output now includes histograms for each sample with superimposed normal distribution curves. It also shows kurtosis, skew, and an omnibus measure of normality for each sample as well as the O’Brien homogeneity of variance test. Explanatory footnotes have been added to the output.
  
  

  Histograms for subgroups of ANOVA

• Spearman’s and Pearson’s correlation output now includes scatterplots and lines of best fit.
  
  

  Scatterplot for assessing linear correlation

• All html reports are portable along with their images (stored in a subfolder of the same name).
• When titles/subtitles are being changed, the rest of the example report table stays the same. This removes an annoying “flicker” effect when typing in titles/subtitles.
• The redundant Clear button has been removed from Statistical Test dialogs.
• An hourglass displays when opening statistics tests and report tables for the first time in case of a brief delay on first use.

There have also been some important and edge-case bug fixes:

• All images are now uniquely named and stored in report-name-based subfolders if “added to report” has been selected, or in the internal folder otherwise. This guarantees the correct images will always be displayed and that saved HTML reports will work.
• The page break in independent t-test output has been repositioned to below the histograms.
• Changing to raw data display, and then changing table source, no longer prevents the example table from displaying.
• Internal footnotes in expanded output now work for Windows users.

0.9.1 is out and there have been a lot of improvements this time:

• All output now displays inside design dialogs. In the case of report tables, there is an option to expand output. This is especially important for displaying larger report tables on netbooks.
• Independent t-test output now includes two histograms with superimposed normal distribution curves.  It also shows kurtosis, skew, and an omnibus measure of normality for each sample as well as the O’Brien homogeneity of variance test. Explanatory footnotes have been added to the output e.g. explaining the p value or what kurtosis means.
  
  

  Independent t-test support graphics

• Guidance given on need to assess normality of each sample when more than one (part of test selection process).
• Hovering over cells in the data entry/editing grid displays appropriate value labels. E.g. hovering over 1 in a gender field may show the tooltip “Male” (if that label has been set up by the user)
• Can update variable details from within data editing/entry grid by right clicking on column labels. This ability is signalled by tooltips.
• Date format (e.g. US) is now automatically extracted from the operating system rather than requiring user preferences. Preferences now sets reporting explanation level (still not operational).

There have also been a few bug fixes:

• Fixed bug in independent t-test where the std dev displayed for sample b was actually that of sample a.
• Now copes with filter on a string variable when creating divider.
• Fixed loss-of-focus bug in Windows when typing titles and subtitles after having clicked html widget.