Statistics Open For All

SOFA stands for Statistics Open For All. What does “Open For All” mean and why is it important?

Open For All means:

• The statistical algorithms used are visible and can be examined by interested users with minimal difficulty (and no Non-Disclosure Agreements required etc).
• The software is available in the languages users need. At this stage, SOFA Statistics is available in English and has been translated into Galician (largely) and Russian (partially) but the goal is to include as many languages as possible.
• The program is available without payment (especially important for students and people in developing nations).
• SOFA Statistics will run on as many computer environments as practical. Currently, only Windows and Ubuntu Linux are supported, but the goal is to add a Mac package ASAP, and possibly some other Linux distributions.
• The program tries to reduce the amount of prior learning a user has to have to use the package successfully and appropriately. It is not assumed that statistics can be used without thought or any statistical insight, but the goal is to help the user make the right decisions at the right points.

So why does this matter?

• So students can easily access useful and educational statistical software (no, a spreadsheet doesn’t count 😉 )
• To allow smaller, or poorly-resourced organisations e.g. non-governmental social service organisations/charities/groups in developing nations etc to conduct basic quantitative research and to generate useful ad hoc and routine reports
• Because statistical thinking is a fantastic intellectual resource that deserves greater appreciation. It is a shame that the main idea most people have about statistics is “Lies, damned lies, and statistics”.

The 0.9 series has finally begun! The main emphasis of this series is to enhance the learn-as-you-go goal of SOFA Statistics. For example, when choosing a statistical test it is important to know if the data is normally distributed or not. You can do this at the point where you’re choosing your test:

Graph to help determine normality

Here is the list of new features added in 0.9.0:

• When selecting a statistical test to use, users can examine the normality of a variable visually and with a test of kurtosis and skew.  The thumbnail can be expanded into a more detailed histogram.
• When opening the project selection dialog it defaults to the version last selected during session.
• Opening the data selection dialog uses the last database and table selected during session.
• Information on filter used also displayed in immediate output.
• The values displayed in tests like the ANOVA take any filtering into account.
• When exporting script, user reminded where it is being appended.
• When editing variable details, user is reminded where the changes are being saved.
• Useful message to user if using deprecated project file format.

Bug fix:

• Fixed simple naming bug which prevented independent t-test and mann-whitney from running.

This is the first release with Matplotlib (scientific graphing) included and the 0.9 series of releases should include lots more graphing and internal documentation to a) help users choose the correct test and b) confirm that they made the right choice.

NB The current plan is to use RaphaelJS for the output charting, not Matplotlib.  Matplotlib graphics will be restricted to a support role, albeit a very important one.  This decision is consistent with the “beautiful output” goal of SOFA Statistics.  RaphaelJS is still maturing as a library, so it may be a while till conventional output graphing is part of SOFA Statistics.  But most of the foundations are in place.

Government investment in open source makes sense as a way of reducing the cost of government service delivery.

The model is this – a government agency invests a sum of money to benefit a government sector e.g. education, health, social service delivery etc. As long as the software is open source, the government sector gets a multiplier effect – any further development of the software in other countries will further benefit education, health, social service delivery etc in the original country.

“This has certainly been New Zealand’s experience with Moodle, a GPL licenced online learning management system. A few years ago Moodle was a nice platform, had a good development community and a few hundred sites. A New Zealand TEC fund was used to add a number of features, not least amongst these were some very solid enterprise and performance capabilities.

Moodle is now used in over 13,000 sites and must be the world’s most popular LMS. Investment from institutions such as the UK’s Open University has now well exceeded NZ’s initial investment. Yet because of the GPL New Zealand still gets to benefit from other peoples’ significant work.” (New Zealand Open Source Society 2007)

Now imagine if there was a hypothetical open source program 😉 that could be used throughout the education sector (senior secondary, polytechs, universities), across a myriad of smaller organisations that supply social services on behalf of government, and across the private sector as well. It enables everything from the teaching of statistics through to routine management reporting. A bit of assistance in its further development could have a large spin-off. Makes you think.