SOFA Stats

One of the biggest changes this release is behind the scenes – the folders SOFA Statistics is stored in have changed from “sofa” to “sofastats” to make them more distinct (there are other sofa programs ;-)). This was something important to get done before the version 1.0 release. Although this is a good change for the long-term it means users upgrading from 0.9.24 and below will need to rename their /home/username/sofa folder to /home/username/sofastats if they wish to work with old data or settings. For Ubuntu users, it is also important to manually uninstall previous versions of SOFA first so that “sofastats” can overwrite the application icon put there originally by “sofa”.

More noticeable, is the change to version upgrade checking. Auto-checking for any version upgrades is now the default. The goal is to help users stay up-to-date with bug fixes, improvements etc. Users can disable this feature by selecting the “No checking” option, or only auto-check for major upgrades. This feature only reads the contents of a small text file on the sofastatistics.com domain containing the latest version number.

Other changes include:

• SOFA suggests names more likely to be valid when user selects a file for importing.
• Can now import csv files with a txt extension.
• MySQL and PostgreSQL can now accept blank passwords.
• Better message for user if comtypes problem.
• Windows and Mac versions will no longer accept being launched with Python 2.7 and will give useful messages about fixing the problem. For Windows, the importance of installing the site-packages to python26 rather than python27 is explained.
• Mac version uses new, easier-to-install library to connect to MySQL databases.

The main series of bug fixes relate to recoding data and making new data tables or redesigning them.

• Fixed numerous recode bugs – mostly to do with application of recoded value labels.
• Fixed bug when making a new table with an existing name. Now stays open to give user choice as to what to do next e.g. change table name.
• Fixed misc bugs when deleting/adding/renaming tables while a database other than the default SOFA database is selected.
• Fixed bug running frequency table using MySQL where the category items are long integers.
• Fixed bug where Data List report selected and then the database is changed.

Ideally, the next release should be version 1.0.0

You can get an overview of SOFA Statistics in just 3 minutes at:

Fast paced, great soundtrack

The video was made with OpenShot, Blender, recordMyDesktop, and the GIMP. The soundtrack is New Day (E-SHINE Remix) by BertycoX.

30,000 downloads and growing

SOFA Statistics has now been downloaded over 30,000 times and the pace has accelerated in recent months. This is encouraging for the project as preparations are made for the version 1.0 release.

Linux Journal review of SOFA Statistics

Version 1.0 of SOFA Statistics is about to be released. This is a big opportunity to introduce SOFA Statistics to a much wider audience.

SOFA Statistics is already receiving positive attention in the open source and Linux community. Linux Journal, for example, recently reviewed SOFA Statistics and ended the piece with “My hope is that this program becomes an adopted industry standard of sorts, mentioned in everyday conversation by organization workers the world over …” (Linux Journal Jan 2011, Issue 201, pp.40-41). This is fantastic support for the project, and SOFA was also fortunate enough to get on the cover of the Ubuntu magazine, Full Circle Magazine:

SOFA Statistics in Full Circle Magazine

The challenge is to get into the wider IT press and beyond so that people looking for a stats/reporting package automatically include SOFA Statistics as one of their options.

Do you have any suggestions? We welcome comments on this topic. A version 1.0 release is an opportunity that shouldn’t be missed. You can either post a message or email me at grant@sofastatistics.com

A key goal of SOFA Statistics is to help users “learn as they go”. The latest version takes a further step in that direction by integrating on-line help into all the statistical tests. In the example below, clicking on the “Help” button in the Chi Square dialog opens the appropriate page in the client’s web browser.

Integrated on-line help for statistical tests

Version 0.9.24 also makes it easier for users to select the right test, and feel confident they have made an appropriate choice, by adding tips for any tests previously lacking any.

Tips for all tests

The other change is that passwords are now hidden when configuring database connections in projects.

There has been an important bug fix for a problem introduced in 0.9.23 which prevented Excel importing.

Other bug fixes have included:

• PostgreSQL and MS SQL Server now cope with databases which are listed but where a connection is not possible e.g. because of permissions.
• Fixed minor bug in sqlite table name test – now commits when making table and before dropping it so a chance to succeed or fail on individual step. Otherwise, net effect is no change (table added and removed) so no chance to fail.
• Safer reading and executing project and variable detail code in Windows (change new lines to required sort).
• Now possible to run test code in Windows (change new lines to required sort).
• Fixed bug when refreshing default sofa tables but not in the sofa sqlite connection.

This release brings SOFA another step closer to the version 1.0 release expected early in 2011.

SOFA Statistics version 0.9.23 makes importing data from spreadsheets (and csv) much more robust. In particular, it copes better with inconsistent data types in a single field and give the user a choice as to which to use:

Choosing data type

The latest version also has a much more reliable method of identifying and extracting date data from ods (OpenOffice Calc, Gnumeric, Google Docs spreadsheets).

Other changes include:

• If no change to table design when clicking Update, get a message that there was no change instead of no response.
• When opening data tables, the read only checkbox is ticked or unticked according to whether showing the demonstration table or not when in the default sofa database.
• Now handles PostgreSQL databases with different schemas.
• When run in console, SOFA produces much more useful input on progress during initial stages and on any error encountered.
• Height setting algorithm for data editing/display grid now copes with taller e.g. double height, taskbars in Windows.
• More reliable method used for identifying the folder location of SOFA.

There are also numerous bug fixes:

• All lines in multi-line text cells are now imported from ods spreasdsheets correctly.
• Errors when getting project settings file at start return useful message and don’t fail because tried to use msgbox before the GUI application is running.
• Better handling of errors when identifying date format. Falls back to a reasonable default.
• Fixed error preventing message being shown when unable to change database details e.g. selecting a different db in the drop-down.
• Fixed bug in legend of auxiliary clustered bar charts when long value labels. Now has automatic line splitting.
• Correct initial display of csv data when importing if missing cells in some rows.
• Majorly reduced impact of bug when creating report tables using numbers with more decimal points than python displays when converting to text e.g. into an SQL statement ready for execution.
• CSV importing now provides useful error messages if actually an Excel spreadsheet saved as a csv with Excel features preserved (i.e. a faulty csv file). Also copes better with faulty delimiters being manually supplied.

SOFA Statistics 0.9.22 is getting very close to the milestones set for a version 1.0 release. The main changes are:

Scatterplots can be run in series e.g. by country.

Scatterplot series

SOFA Statistics now checks for upgrades if so desired:

Upgrade option

And there is much better support for translations (English, Croatian, Spanish, Russian, Galician, and Breton so far):

Croatian Translation

If you want to help translate SOFA Statistics into your own language, you can get started at SOFA Statistics Translations:

Easy to help with translation

Additional changes are:

• Less alarming message when PostgreSQL not installed on a system.
• Better handling of python version mismatch taking OS into account.
• Better help text for Report Tables and Data Entry/Editing menu choice.
• Warning about changes to default report on version change now appear on top of main start screen not before it.

There are also some bug fixes:

• Area charts get legend back.
• Fixed bug stopping useful error messages appearing when trouble loading first frame.
• Wider config dialog in Report Tables for Row Stats to show all text e.g. Standard Deviation.

SOFA Statistics now lets you make a series of charts by a selected variable. For example, if you have Pie Charts of user satisfaction you can make one for each region automatically. All chart types can be made into series apart from scatter-plots.

Chart series example

Several of the help images on the main form have also been improved:

Improved help images example

Other improvements include:

• Better printing of output reports through web browsers e.g. Firefox.
• Imports numeric fields in csv files as numbers even if using commas as decimal separator (e.g. as in Germany).
• Shifted Help button to top left of main dialogs.
• Removed Export button (from view).
• Changed text in Run button to Show Results.
• Mann-Whitney U now explicitly says it is one-tailed.

To assist with the promotion of SOFA Statistics, a discreet “sofastatistics.com” text is now added to edge of each chart.

Additionally there have been some important bug fixes:

• Fixed page break problem when printing.
• Major bug fix for Windows users with certain user names. Fixed bug in Windows version when username combined with a backslash equals a character e.g. \admin. The bug affected css files and all charts and stats tests.
• Fixed bug when database engine returns non-integers for frequencies e.g. 2l (for long number type) instead of 2.
• Fixed vertical positioning problem for report table and stats select dialogs on 768 high resolutions (Windows at least).
• Fixed bug providing user feedback when problems with css files.
• Fixed y-axis title bug.

SOFA Statistics is very close now to the version 1.0 release.

Version 0.9.20 of SOFA Statistics brings a number of improvements …

• Legend closer to charts and outer background colour now covers the x-axis labels.
• Better placement of y-axis titles.
• SOFA warns user if not enough suitable variables for an analysis – e.g. paired t-test – and suggests a likely cause and solution.
• Default sofa_db row in projects is read-only.
• Better message to user if attempting to connect to an empty SQLite database.

… and fixes:

• Fixed bug preventing projects being saved.
• Fixed bug stopping charts working properly with some database engines.
• Can now handle multiple SQLite and MS Access databases successfully.
• Fixed bug preventing Preferences button being clicked.