User Tools

Site Tools


proj:wininstallers

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
proj:wininstallers [2019/10/05 18:16]
114.23.208.115
proj:wininstallers [2022/08/14 05:15] (current)
151.210.130.23 [Set launch.py dependency imports]
Line 3: Line 3:
 [[:​proj:​project | Project]] [[:​proj:​project | Project]]
  
-====== Windows ​NSIS Installer Notes ======+====== Windows Installer Notes ====== 
 + 
 +===== What Next ===== 
 + 
 +If already set up dev pipeline on machine go straight to "Make Package - Overview"​. 
 + 
 +Otherwise, "​Windows Development and Packaging - From Scratch"​
  
 ===== Make Package - Overview ===== ===== Make Package - Overview =====
Line 11: Line 17:
   * git pull (SSH passphrase "​matilda loves linux better"​)   * git pull (SSH passphrase "​matilda loves linux better"​)
   * bring across licence.txt if changed   * bring across licence.txt if changed
-  * Copy contents of D:​\sofa_dev_win\sofastatistics\sofastats over top of D:​\sofa_dev_win\packaging\sofastats (leaving behind tests, eclipse project files, __pycache__ folders, and any exes such as convert.exe and wkhtmltopdf.exe)+  * Copy contents of D:​\sofa_dev_win\sofastatistics\sofastats over top of D:​\sofa_dev_win\sofa4packaging\sofastats (leaving behind tests, eclipse project files, __pycache__ folders, and any exes such as convert.exe and wkhtmltopdf.exe)
   * Eliminate debug = True (esp dev_debug = True in home.py)   * Eliminate debug = True (esp dev_debug = True in home.py)
-  * Delete build and dist folders under D:​\sofa_dev_win\packaging\sofastats (to ensure clean build)+  * Delete build and dist folders under D:​\sofa_dev_win\sofa4packaging\sofastats (to ensure clean build)
   * Create executable   * Create executable
  
-  cd D:​\sofa_dev_win\packaging\sofastats\+  cd D:​\sofa_dev_win\sofa4packaging\sofastats\
   pyinstaller sofastats.spec   pyinstaller sofastats.spec
  
-  * Under D:​\sofa_dev_win\packaging\sofastats\ copy everything but __pycache__,​ build, and dist into dist\sofastats. +  * Under D:​\sofa_dev_win\sofa4packaging\sofastats\ copy everything but __pycache__,​ build, and dist into dist\sofastats. 
-  * Change the version at the top of In D:​\sofa_dev_win\1 sofastats.nsi:​+  * Change the version at the top of In D:​\sofa_dev_win\packaging\1 sofastats.nsi:​
  
-  !define SOFA_VERSION "1.5.3"+  !define SOFA_VERSION "1.5.4"
  
-  * In D:​\sofa_dev_win\ right click on  "1 sofastats.nsi"​ and "​compile NSIS script"​ then click on "Test Installer"​ button.+  * In D:​\sofa_dev_win\packaging ​right click on  "1 sofastats.nsi"​ and "​compile NSIS script"​ then click on "Test Installer"​ button.
   * Run SOFA from icon on Desktop   * Run SOFA from icon on Desktop
   * (Assuming OK) copy sofastats-x.x.x_for_win.exe to the USB for safe-keeping   * (Assuming OK) copy sofastats-x.x.x_for_win.exe to the USB for safe-keeping
   * Compress to zip folder so sofastats-x.x.x_for_win.zip   * Compress to zip folder so sofastats-x.x.x_for_win.zip
  
-===== Windows Development and Packaging =====+===== Windows Development and Packaging ​- From Scratch ​=====
  
 In short, how we get from a clean Windows machine to a development (to fix Windows-specific bugs) and packaging environment for SOFA. In short, how we get from a clean Windows machine to a development (to fix Windows-specific bugs) and packaging environment for SOFA.
Line 40: Line 46:
  
   * **sofastatistics** (actually, don't make it directly - instead, add it when cloning the git repo). This is the code. Can make changes here to get SOFA working on Windows and push the changes up into origin. Update this folder then copy across contents to sofa4packaging/​sofastats to keep the version to be packaged up-to-date   * **sofastatistics** (actually, don't make it directly - instead, add it when cloning the git repo). This is the code. Can make changes here to get SOFA working on Windows and push the changes up into origin. Update this folder then copy across contents to sofa4packaging/​sofastats to keep the version to be packaged up-to-date
-  * **dependencies** - binaries such as wkhtmltopdf.exe and specific dependencies of those binaries e.g. delegates.xml (needed by convert.exe) +  * **dependencies** - binaries such as wkhtmltopdf.exe and specific dependencies of those binaries e.g. delegates.xml (needed by convert.exe). You'll find them under sofa_win/​sofa_dev_win/​dependencies on the Ubuntu dev machine. Keep the Ubuntu dev machine versions up-to-date with what is used on the Windows machine when developing there and running SOFA unpackaged. 
-  * **packaging** - put all the files managing the actual building of packages e.g. spec files, nsi files etc. +  * **packaging** - put all the files managing the actual building of packages e.g. spec files, nsi files etc. You'll find them under sofa_win/​sofa_dev_win/​packaging on the Ubuntu dev machine
-  * **sofa4packaging** - put contents of sofastatistics ​under the packaging/​sofastats ​folder and add launch.py, import2run.py,​ and sofastats.spec as well from /​packaging. +  * **sofa4packaging** - put contents of sofastatistics folder and add launch.py, import2run.py,​ and sofastats.spec as well from /packaging. Leave behind __pycache__,​ app.fil and sofastats.pot
-  * **store** - put junk here to avoid polluting / over-complicating other folders+  * **store** - put junk here to avoid polluting / over-complicating other folders
  
 ==== Python and dependencies installed ==== ==== Python and dependencies installed ====
  
-Set up prerequisites for running SOFA working on Windows at all - install Python 3.(https://​www.python.org/​downloads/​) ​and all required pip and system dependencies (perhaps PostgreSQL). See list of dependencies further down. Allow Python ​to be set in PATHNo need to specify full path to python exe every time.+Set up prerequisites for running SOFA working on Windows at all - install ​latest version of Python ​3 e.g. 3.(https://​www.python.org/​downloads/​). Allow Python to be set in PATH so don't need to specify full path to python exe every time. Allow to be installed for all users. Also install ​all required pip and system dependencies (perhaps PostgreSQL). See list of dependencies further down under "Set launch.py dependency imports"​. Note pip install Pillow not PIL; matplotlib not pylab; pywin32 ​to enable associated imports (and https://​github.com/​KBNLresearch/​iromlab/​issues/​100) etc. Have to translate from import requirements ​to module requirements. Usually straight forward.
  
 Target pip installations at the python you will be building against (possibly only one). No need to be admin for pip installations unless Python was installed as admin. Target pip installations at the python you will be building against (possibly only one). No need to be admin for pip installations unless Python was installed as admin.
Line 53: Line 59:
 If installed locally you might find Python somewhere like: If installed locally you might find Python somewhere like:
  
-C:​\Users\Giles\AppData\Local\Programs\Python\Python37-32+C:​\Users\Giles ​Paton-Simpson\AppData\Roaming\Python\Python39
  
 Run Run
Line 63: Line 69:
   python -m pip install requests   python -m pip install requests
  
-Re: PostgreSQL: use weak password for throw-away PG 11 installation on Win 10.+Re: PostgreSQL: use weak password for throw-away PG latest (e.g. 13) installation on Win 10.
  
 ==== Binary dependencies ==== ==== Binary dependencies ====
 +
 +Put copies of every dependency under D:​\sofa_dev_win\dependencies and also D:​\sofa_dev_win\packaging\sofastats
  
 === wkhtmltopdf (for HTML to PDF) === === wkhtmltopdf (for HTML to PDF) ===
  
-wkhtmltopdf.exe - download installer from https://​wkhtmltopdf.org/​downloads.html then go to installed location e.g. C:\Program Files\wkhtmltopdf and put copy in D:​\sofa_dev_win\store and under D:​\sofa_dev_win\packaging\sofastats+wkhtmltopdf.exe - download installer from https://​wkhtmltopdf.org/​downloads.html then go to installed location e.g. C:\Program Files\wkhtmltopdf and put copy in D:​\sofa_dev_win\store and under D:​\sofa_dev_win\sofa4packaging\sofastats
  
 === convert (for PDF to PNG) === === convert (for PDF to PNG) ===
  
-convert.exe - install the PORTABLE version of Imagemagick ​(ticking the legacy components part or else no convert.exe for you!) - e.g. ImageMagick-7.0.8-67-portable-Q16-x64.zip.+convert.exe - unzip / install the PORTABLE version of Imagemagick e.g. ImageMagick-7.0.10-47-portable-Q16-x64.zip ​into D:​\sofa_dev_win\store\imagemagickNaming convention explained below:
  
   Versions with Q8 in the name are 8 bits-per-pixel component (e.g. 8-bit red, 8-bit green, etc.),   Versions with Q8 in the name are 8 bits-per-pixel component (e.g. 8-bit red, 8-bit green, etc.),
Line 85: Line 93:
 Also need delegates.xml Also need delegates.xml
  
-=== ghostscript (what convert delegates specific PDF to PNG tasks to) ===+Note - delegates.xml must be edited so convert.exe can find the correct ​ghostscript ​executable namely gswin64c.exe ​(otherwise it defaults ​to gswin32c.exe). The ghostscript executable will be in the same folder as convert.exe so we don't need to worry about path thankfully.
  
-gswin64c.exe and gsdll64.dll from https://​www.ghostscript.com/​download/​gsdnld.html +Replace:
-==== Get dev toolchain working e.g. eclipse, git ====+
  
-Install git and git bash etc - https://​gitforwindows.org+    <​delegate decode="​ps:alpha" stealth="​True"​ command="&​quot;​@PSDelegate@&​quot;​ ...
  
-Generate SSH key using Git Bash GUI+with:
  
-Change remote as required so has ssh in it.+    <​delegate decode="​ps:​alpha"​ stealth="​True"​ command="&​quot;​gswin64c.exe&​quot;​ ...
  
-Use OpenSSH-generated key made for Windows machine. Don't try to reuse Linux one - bad practice.+=== ghostscript (what convert delegates specific PDF to PNG tasks to) ===
  
-Extras if paint self in corner with SSH key:+gswin64c.exe and gsdll64.dll from https://​www.ghostscript.com/​download/​gsdnld.html 
 +==== Get dev toolchain working e.g. eclipse, git ====
  
-https://stackoverflow.com/​questions/​50918607/​remove-cached-password-for-git-on-windows+Install git and git bash etc - https://gitforwindows.org
  
-Reinstalling won'​t ​solve it. You will have to update it in your Credential Manager.+Generate SSH key using Git GUI (Git Bash). Use passphrase "​matilda loves linux better"​ (also in keepass2). Use OpenSSH-generated key made for Windows machine. Don'​t ​try to reuse Linux one - bad practice.
  
-Go to Control Panel > User Accounts > Credential Manager > Windows Credentials.+Change git remote as required so has SSH in it. id_rsa.pub is somewhere like C:​\Users\Giles Paton-Simpson\.ssh. Log into launchpad (launchpad@p-s.co.nz using pwd from keepass2) and add additional SSH key.
  
-You will see Git credentials in the list (e.g. git:​https://​...).+Extras if paint self in corner with SSH key: https://​stackoverflow.com/​questions/​50918607/​remove-cached-password-for-git-on-windows Reinstalling won't solve it. You will have to update it in your Credential Manager. I.e. go to Control Panel > User Accounts > Credential Manager > Windows Credentials. ​You will see Git credentials in the list (e.g. git:​https://​...). Click on it, update the password, and execute git pull/push command from your Git bash and it it won't throw any error message.
  
-Click on it, update the password, and execute ​git pull/push command from your Git bash and it it won't throw any error message.+Then get latest SOFA code using git within ​Git Cmd:
  
-  ​cd D:\sofa_dev_win+  D: 
 +  cd sofa_dev_win
   git clone https://​git.launchpad.net/​sofastatistics   git clone https://​git.launchpad.net/​sofastatistics
  
 ==== Get SOFA working on Windows ==== ==== Get SOFA working on Windows ====
  
-  cd "​D:​\sofa_dev_win\sofastatistics\sofastats+  cd "​D:​\sofa_dev_win\sofastatistics\"​ 
-  python start.py+  python ​-m sofastats.start 
 +  
 +Note - SOFA puts parent path into sys.path so we start in the same folder as start.py
  
 Iron out inevitable bugs. Iron out inevitable bugs.
Line 134: Line 145:
 ==== Set launch.py dependency imports ==== ==== Set launch.py dependency imports ====
  
-Run special script ​<specify properly here> ​to identify actual dependencies so they can be called by launch.py+Run special script ​sofa_dev_win/​packaging/​import_dependencies.py ​to identify actual dependencies so they can be called by launch.py
  
   #import CUBRID-Python ## 2014 Python 2 only   #import CUBRID-Python ## 2014 Python 2 only
-  import PIL  ## actually ​pillow ​installed+  import PIL  ## actually ​Pillow ​installed
   import PyPDF2   import PyPDF2
-  import adodbapi+  import adodbapi ​ ## part of pywin32 now
   import agw   import agw
   import numpy   import numpy
Line 146: Line 157:
   import psycopg2   import psycopg2
   import pylab  ## part of matplotlib   import pylab  ## part of matplotlib
 +  import pyodbc
   import pymysql   import pymysql
   import requests   import requests
Line 176: Line 188:
 Otherwise errors like: ModuleNotFoundError:​ No module named '​wx.lib.agw.hypertreelist'​ Otherwise errors like: ModuleNotFoundError:​ No module named '​wx.lib.agw.hypertreelist'​
  
-==== Identify Windows C Runtime (CRT) dependency path ====+==== Identify Windows C Runtime (CRT) dependency path (nothing to install) ​====
  
 This dependency is required if using Python >= 3.5 and Windows systems < Windows 10. I.e. it is required. This dependency is required if using Python >= 3.5 and Windows systems < Windows 10. I.e. it is required.
Line 193: Line 205:
  
 ==== Run pyinstaller for full run (making spec and using) ==== ==== Run pyinstaller for full run (making spec and using) ====
 +
 +Note - sofa_win_dev/​sofa4packaging/​sofastats must contain latest code plus dependencies plus launch.py, import2run.py,​ and sofastats.spec as well from /packaging. Leave behind pycache, app.fil and sofastats.pot.
  
 Note - usually people only use the --onedir option to start with for debugging reasons and the --onefile option later. Note - usually people only use the --onedir option to start with for debugging reasons and the --onefile option later.
Line 202: Line 216:
 The actual command: The actual command:
  
-  cd C:​\sofa_dev_win\packaging\sofastats\+  cd D:​\sofa_dev_win\sofa4packaging\sofastats\
   pyinstaller --clean --exclude start --path C:​\Windows\WinSxS\x86_microsoft-windows-m..namespace-downlevel_31bf3856ad364e35_10.0.17134.1_none_50c6cb8431e7428f launch.py   pyinstaller --clean --exclude start --path C:​\Windows\WinSxS\x86_microsoft-windows-m..namespace-downlevel_31bf3856ad364e35_10.0.17134.1_none_50c6cb8431e7428f launch.py
  
-The binary executable is C:​\sofa_dev_win\packaging\sofastats\dist\launch\launch.exe+The binary executable is D:​\sofa_dev_win\sofa4packaging\sofastats\dist\launch\launch.exe
  
-Shift all C:​\sofa_dev_win\packaging\sofastats\ in (except for __pycache__,​ build, or dist) and rename launch folder to sofastats so calls to sofastats package work.+Shift all D:​\sofa_dev_win\sofa4packaging\sofastats\ in (except for __pycache__,​ build, or dist) and rename launch folder to sofastats so calls to sofastats package work.
  
 ==== Rename, relocate, and edit spec file ==== ==== Rename, relocate, and edit spec file ====
Line 213: Line 227:
 Warning - must wipe build folder first for a clean exe build Warning - must wipe build folder first for a clean exe build
  
-ghostscript - use the commandline version of ghostscript. http://​stackoverflow.com/​questions/​2598669/​ghostscript-whats-are-the-differences-between-linux-and-windows-variants. On Windows you have two executables, ​gswin32c.exe and gswin32.exe instead of gs only. The first one is to run Ghostscript on the commandline ("DOS box"), the second one will open two GUI windows: one to render the output, another one which is console-like and shows GS stdout/​stderr or takes your command input if you run GS in interactive mode.+ghostscript - use the commandline version of ghostscript. http://​stackoverflow.com/​questions/​2598669/​ghostscript-whats-are-the-differences-between-linux-and-windows-variants. On Windows you have two executables, ​gswin64c.exe and gswin64.exe instead of gs only. The first one is to run Ghostscript on the commandline ("DOS box"), the second one will open two GUI windows: one to render the output, another one which is console-like and shows GS stdout/​stderr or takes your command input if you run GS in interactive mode.
  
-ImageMagick - use the portable version (big fat convert.exe with everything ​bakes in vs scrawny non-portable version which relies the dependencies)+ImageMagick - use the portable version (big fat convert.exe with everything ​baked in vs scrawny non-portable version which relies the dependencies)
  
   binaries=[   binaries=[
-      ('C:/Program Files/​wkhtmltopdf/bin/​wkhtmltopdf.exe',​ '​.'​), ​ ## the docs for 3.4 show the structure as (binary, subfolder) where '​.'​ means the main folder +      ('D:/sofa_dev_win/dependencies/​wkhtmltopdf.exe',​ '​.'​), ​ ## the docs for 3.4 show the structure as (binary, subfolder) where '​.'​ means the main folder 
-      ('C:/​sofa_dev_win/​ImageMagick-7.0.8-44-portable-Q16-x64/​convert.exe',​ '​.'​), ​ ## need portable version +      ('D:/​sofa_dev_win/​dependencies/​convert.exe',​ '​.'​), ​ ## need portable version 
-      ('C:/Program Files (x86)/​gs/​gs9.27/bin/gswin32c.exe', '​.'​),​ +      ('D:/sofa_dev_win/dependencies/gswin64c.exe', '​.'​),​ 
-      ('C:/Program Files (x86)/​gs/​gs9.27/bin/gsdll32.dll', '​.'​),​+      ('D:/sofa_dev_win/dependencies/gsdll64.dll', '​.'​),​
   ],   ],
   datas=[   datas=[
-      ('C:/​sofa_dev_win/​ImageMagick-7.0.8-44-portable-Q16-x64/​delegates.xml',​ '​.'​), ​+      ('D:/​sofa_dev_win/​dependencies/​delegates.xml',​ '​.'​), ​
   ],   ],
  
-exe - icon='C:​\sofa_dev_win\sofa_32x32.ico'​+exe - icon='D:​\sofa_dev_win\packaging\sofa_32x32.ico'​
  
 exe - upx=False exe - upx=False
Line 251: Line 265:
 exe - debug=False exe - debug=False
  
-Put in C:​\sofa_dev_win\packaging\sofastats\ as sofastats.spec. Needs to be in same folder as scripts being packaged i.e. launch.py etc+Put in D:​\sofa_dev_win\sofa4packaging\sofastats\ as sofastats.spec. Needs to be in same folder as scripts being packaged i.e. launch.py etc
  
 ==== Run pyinstaller off spec file ==== ==== Run pyinstaller off spec file ====
  
-  cd C:​\sofa_dev_win\packaging\sofastats\+  cd D:​\sofa_dev_win\sofa4packaging\sofastats\
   pyinstaller sofastats.spec   pyinstaller sofastats.spec
  
-Shift all C:​\sofa_dev_win\packaging\sofastats\ in (except for __pycache__build, or dist)Folder already named sofastats ​because of spec edits. +Note - sofa_win_dev/​sofa4packaging/​sofastats ​must contain latest code plus dependencies plus launch.pyimport2run.py, and sofastats.spec as well from /packaging. Leave behind pycache, app.fil and sofastats.pot.
 ==== Test binary and SOFA code folders on clean machine ==== ==== Test binary and SOFA code folders on clean machine ====
  
Line 267: Line 280:
 ==== Make NSIS installer ==== ==== Make NSIS installer ====
  
-Use UltraModernUI+Install NSIS https://​nsis.sourceforge.io/​Download
  
-Get latest version from [[https://​sourceforge.net/​projects/​ultramodernui/​files/​UltraModernUI/​]]+Use UltraModernUI - get latest version from [[https://​sourceforge.net/​projects/​ultramodernui/​files/​UltraModernUI/​]]
  
 Note - don't need to specify every file - can just use the recursive syntax Note - don't need to specify every file - can just use the recursive syntax
proj/wininstallers.1570313798.txt.gz · Last modified: 2019/10/05 18:16 by 114.23.208.115