Statistics Open For All

The latest version of SOFA Statistics finally delivers some visual eye candy in the form of dynamic bar charts. These look good, print cleanly, can be shared easily (anyone with a web browser can view them), and there is lots of scope for further refinement and customisation.

SOFA charts are dynamic and respond when you move the mouse over them. Unfortunately, the charting library (Dojo) doesn’t work well with WordPress, so a static snapshot has been included in this blog post instead. A live example can be found in the SOFA features page.

Static image of SOFA Bar Chart

SOFA Bar Charts dialog

Version 0.9.15 also copes properly with upgrades e.g. on top of 0.9.14. You can download it from here: http://www.sofastatistics.com/downloads.php

Coming versions of SOFA will add new chart types (probably Pie charts and Clustered Bar Charts first) and the ability to style and otherwise configure them.

Version 0.9.14 has three significant new features:

1) Row stats tables now include min, max, and range.

Min, Max, and Range

2) Report tables now visibly separate row variables:

Visible separation

3) CSV importing has been substantially improved. User gets option of confirming or changing auto-identified csv file delimiter and encoding. Confirming presence of a header is also integrated and the first 20 rows are displayed in a table with visible cell borders.

CSV import options

The full list of changes is:

• Added min, max, and range to row stats tables.
• Report tables are much easier to read when there are multiple row variables. Added pale line between row variables in left-most column.
• CSV importing has been substantially improved. User gets option of confirming or changing auto-identified csv file delimiter and encoding. Confirming presence of a header is also integrated and the first 20 rows are displayed in a table with visible cell borders. The import is more forgiving of some faults e.g. missing items at the end of a row. These are automatically filled with empty strings. When errors are encountered in csv import SOFA tells user the relevant line number. Also made other minor improvements to (error) messages from csv importing.
• Added Filter button next to Table dropdown to make filtering functionality of SOFA more accessible.
• Much simpler installation for Ubuntu/Linux Mint users. No additional steps required now that SOFA has started using gtk-webkit.
• ANOVAs and similar group by analyses now cope with massive tables when selecting unique values to show in group by drop downs.
• Appropriate messages are shown to users when large tables underlying analyses are about to be run. The user can opt out.
• Changed default for Anova from precision to speed.
• Better messages when chopping group by values shown.
• If SOFA has been opened by an incompatible version of Python it will give a useful message and leave a helpful text file behind with instructions for fixing the problem.
• Better messages when Chi Square warnings about number of rows, cols, or cells.
• Better tool tip for setting MS SQL Server host e.g. my-svr-01, 1433.
• Added minor improvement to tool tips for data tables. It now always shows labelled value if possible and read only status as suffix as appropriate.
• Created more robust startup process with better error messages. Users also able to run test code (e.g. sent by the SOFA developer) and get result. Great for diagnosis.
• Overhauled start code so checks version if possible, updates version, and makes a recovery folder if not current.

There are also some important bug fixes:

• Fixed bug where only one of multiple databases shown e.g. for MySQL, PostgreSQL, and MS SQL Server.
• Fixed bug stopping MS SQL Server 2005 from opening.
• MS SQL Server 2000 will now open successfully, even if master is not the default table.
• Fixed bug with Chi Square on MySQL caused by dividing Decimal data type by float.
• Hourglass starts earlier when saving filter options.
• Fixed minor bug with display tool tips in read-only tables.
• Now using gtk-webkit inside wxpython. Now displays background images in tables properly in Ubuntu/Linux Mint. Can start working on all output charting.

SOFA Statistics now comes with integrated online help. Clicking on the “Online Help” button opens the SOFA Statistics User Guide in the user’s default browser.

Integrated Online Help

The goal is to make the user documentation as useful as possible – both to new users and to users trying to solve specialised problems. Over time, lots of gaps will be filled so it will be worth making repeat visits to the site.

The User Guide

Here is the full list of new features:

• Now able to look at and manipulate your SOFA data using free and open source tools like Sqlite Database Browser
• Improvements for connecting to Google Docs spreadsheets
• Major improvements to usability around data entry
• Now able to edit variable details from the table config dialog
• Much more useful help when errors encountered importing csv files

Version 0.9.12 also fixes numerous bugs with the Google Docs spreadsheet

The latest version, 0.9.11, can import data from your on-line Google Docs Spreadsheets. I will supply more details later, but it is easy to conduct on-line surveys with Google Spreadsheets underneath. And now you can analyse those results using SOFA Statistics.

To connect to your Google account, there is a new button on the standard Import dialog.

New Google Spreadsheet button

This opens a simple dialog letting you sign into your Google account. The existing spreadsheets are listed and then you can select a worksheet. Once this is downloaded, it can be imported in the standard SOFA manner.

Download Google Spreadsheet

Version 0.9.11 also makes it easier to work with read-only columns in data tables.

Read-only Columns

The final improvement relates to filtering. If you have trouble with the flexible data filtering option, more helpful examples of correct syntax are supplied.

The main bug fix is that SQLite and MySQL now handle fields containing double quotes.

In less than a week the Mac package for SOFA Statistics has been downloaded over 100 times, representing nearly a quarter of all downloads of version 0.9.10.

Example output on OS X

A positive response was expected given recent survey results at the main SOFA Statistics website – 35% of respondents wanted the Mac package as the next thing to add to the project. And about 14% of visitors to the main website are using a Mac OS. But it was still nice to see the response.

SOFA Statistics has been packaged for Mac OS X (Leopard and Snow Leopard). It only seems right that an application emphasising ease-of-use and beautiful output should be available for Apple users.

SOFA Statistics on OS X

The installer is supplied as a mpkg (metapackage) file inside a disk image. Open the mpkg to install.

SOFA mac installer

The same application now works much the same on Windows XP, Vista, and Windows 7, Ubuntu/Linux Mint, and Mac OS X Leopard and Snow Leopard.

SOFA controls on mac

Even if you are not a Mac user, there are a few extras in the new version:

• MySQL plugin now works with older versions of MySQL e.g. 4.1.
• MySQL now allows port details e.g. for remote connections.
• All database plugins allow empty default database or table configuration details (in which case they’ll use the first).
• Easier to fill in connection details in project. Tooltips (with examples) have been added to database configuration controls to make it more clear what is required from the user.
• Better placement of dialogs on smaller screens so nothing off-screen.
• Problems with connection details raise useful message for users.
• SOFA gives useful message if the SOFA application path is not found because of an unexpected folder name e.g. ‘sofa stats’ rather than ‘sofa’. The Windows installer was also upgraded to give better guidance.
• Better error messages if anything goes wrong with loading of initial images when starting SOFA.

There are also a few bug fixes:

• Fixed bug in calculation of total frequency across rows so that missing values are excluded.
• Background images for tables now available in Ubuntu install.
• Fixed some minor background bugs.

You can download version 0.9.10 now: SOFA downloads

NB 0.9.10 (zero point nine point ten) is newer than 0.9.9.

SOFA Statistics has now reached the 10,000 download milestone on Sourceforge. And the rate of downloading has been steadily increasing. In May 2009 there were only 30 downloads whereas in May 2010 there are likely to be over 50 times that. The main area of work at the moment is creating a Mac package. Follow this blog or the twitter feed (http://twitter.com/sofastatistics) to stay informed.

10000 downloads!

SOFA Statistics is growing in popularity, if SourceForge downloads are any indication.

Sourceforge Download Trend

Of course it is early days yet for the project, with version 1.0 not expected until later this year, but current trends are encouraging. It seems there is a gap in the market for an open source statistics, analysis, and reporting package aimed at non-specialist statisticians.