As SOFA Statistics has gained more functionality it has grown in complexity – there are modules for reading Excel spreadsheets, connecting to Google Docs spreadsheets, displaying charts, displaying GUI widgets etc. Trying to make a single executable for Windows users was always going to be a challenge and would probably involve a lot of trial and error. So it proved.
But there was one technique I used to make the seemingly impossible task manageable. I made a single python script I called launch.py which was responsible for importing all the main modules the executable would need to handle (e.g. matplotlib, MySQLdb etc). I identified the imports I would need by looking at each and every main module in SOFA and adding any external library module imports not already included.
The process of making an executable failed initially, so by variously commenting and uncommenting parts of the launch script I was able to isolate problem modules and fix them. To get PostgreSQL working, for example, I needed to add the following fix:
1 2 3 4 5 6 7 8 9 10 11 | try:
os.environ['PATH'] += ";C:\\Program Files\\PostgreSQL\\9.1\\bin"
import pgdb
except ImportError, e:
pass
|
Here is the full text of launch.py:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 | from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import cgi
import codecs
from collections import defaultdict
from collections import namedtuple
import copy
import csv
import datetime
import decimal
import gettext
import glob
import locale
import math
from operator import itemgetter
import os
import platform
import pprint
import random
import re
import shutil
import socket
import subprocess
import sys
import time
import traceback
from types import IntType, FloatType, ListType, TupleType, StringType
import warnings
import weakref
import webbrowser
import xml.etree.ElementTree as etree
import zipfile
import MySQLdb as mysql
try:
os.environ['PATH'] += ";C:\\Program Files\\PostgreSQL\\9.1\\bin"
import pgdb
except ImportError, e:
pass
import sqlite3 as sqlite
if not hasattr(sys, 'frozen'):
import wxversion
wxversion.select('2.8')
import wx
import wx.lib.iewin as ie
import wx.gizmos
import wx.grid
import wx.html
try:
from agw import hyperlink as hl
except ImportError:
import wx.lib.agw.hyperlink as hl
import numpy as np
import matplotlib
import wxmpl
import pylab
import adodbapi
import pywintypes
import win32api
import win32con
import win32com
import win32com.client
import dao36_from_genpy
import import2run
|
The code for SOFA is cross-platform and I start the Windows packaging process by copying everything across from Ubuntu. It is important in such a case to wipe all pyc files so that platform-specific ones are created for Windows and included in the executable creation process.
The final import statement is for import2run.py. This means that the executable doesn’t hardwire anything beyond the imports. As it happens I started by having import2run contain just the following line:
raw_input("Success!!")
Later, once all the basic imports were working, I changed it to:
import start
to actually load SOFA. NB the executable created using the technique described here doesn’t replace all the SOFA modules with a single executable – its purpose is to replace Python and all the extra libraries such as matplotlib. So the exe is expected to live in the main SOFA program folder (usually in C:\Program Files\sofastats) alongside the usual modules such as core_stats.py. If a user actually had Python 2.6 and all the libraries installed they could either use the exe or run start.py directly themselves. It would have the same effect.
Getting matplotlib to work took a while and involved many false leads. In the end the solution was to copy the entire mpl-data folder (from somewhere like C:\Python26\Lib\site-packages\matplotlib) into the same folder as the sofastats.exe was going to end up.
Some final things I learned about Pyinstaller. –onedir is the default and adds the coll = COLLECT(…) part of the spec file. If making manual changes remember that if you want the onedir approach, don’t include a.binaries in the EXE(…) part and exclude_binaries should be True. If, like myself you want a single executable file, don’t bother with coll = COLLECT(…), include a.binaries, and set exclude_binaries to False. And while testing set debug=True and Console=True so you can see what is going wrong as you refine your spec file, launch.py script etc.
Although GUI2EXE is a wonderful program some aspects may not be compatible with Pyinstaller 1.5.1 so I now build my spec file using makespec.py with the –onefile argument. It works in its basic vanilla form for SOFA using launch.py. You can export the spec file GUI2EXE makes and see the differences.
Here is the final spec file I used:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | a = Analysis([os.path.join(HOMEPATH,'support\\_mountzlib.py'), os.path.join(HOMEPATH,'support\\useUnicode.py'), 'C:\\sofastats_build_exe\\sofa.main\\launch.py'],
pathex=['C:\\Python26\\pyinstaller-1.5.1'])
pyz = PYZ(a.pure, level=9)
exe = EXE( pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
exclude_binaries=False,
name='C:\\sofastats_build_exe\\sofa.main\\sofastats.exe',
debug=True,
strip=False,
upx=False,
console=True )
|
Before going live switch debug and console to False.
This post is largely specific to SOFA Statistics but hopefully it includes some tips which might save others a lot of fruitless struggle. If you have trouble, I found the pyinstaller mailing list people helpful.
