.gitignore

@@ -13,3 +13,7 @@ setup.py
 *.egg-info
 last_params
 last_params.yml
+
+\.idea/
+
+dstat_interface/core/experiments/__pycache__/

CHANGELOG

+Version 1.4.6
+    -Fixed data output for SWV/DPV forward/reverse current
+    -Working progress bars
+    -Refactored a lot of plotting code
+    -Calibration should work properly now
+    
+Version 1.4.5
+    -Made board definitions modular
+    -Fix several bugs with experiment parameters
+    -Uses DAC unit based parameters now (**REQUIRES dstat-firmware@9e4a9f or higher**)
+    -Change package import structure again (main must always be run as module now)
+    -Workaround for weird Gtk+3 redrawing bug on Windows
+
+Version 1.4.4
+    -Make connection code more robust
+    -Execute button disabled until DStat is ready to start
+    -Supports new firmware version strings added in dstat-firmware@c5f9701
+    -Experimental firmware upgrade tool (see DStat Menu)
+    -Fix many bugs
+
+Version 1.4.3
+    -Fix another critical bug with Windows multiprocessing
+    -Allow normal exit even if DStat was never connected
+    -Store last parameters in user folder
+
+Version 1.4.2
+    -Refactor to fix critical bug preventing running packaged versions.
+
+Version 1.4.1
+    -Fixed voltage axis orientation for LSV, CV, SWV, and DPV (Thanks to Dan Bizzotto @ UBC)
+    -Tweaked paver files to make version detection work without git.
+
 Version 1.4
     -Switched to GTK+3
     -Support new DStat communications protocol (requires dstat-firmware>fe50c38)

MANIFEST.in

 include RELEASE-VERSION
 include version.py
 include setup.py
+include main.py
 include paver-minilib.zip
+include LICENSE
+include CHANGELOG
+include README.markdown
+include core/utils/RELEASE-VERSION
 recursive-include dstat_interface *
 recursive-exclude dstat_interface *.pyc
 recursive-exclude dstat_interface *~
+recursive_exclude core last_params.yml
+recursive-exclude . .DS_Store
\ No newline at end of file

README.markdown

 ##### _DStat is described in detail in [Dryden MDM, Wheeler AR (2015) DStat: A Versatile, Open-Source Potentiostat for Electroanalysis and Integration. PLoS ONE 10(10): e0140349. doi: 10.1371/journal.pone.0140349](http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0140349) If you use this information in published work, please cite accordingly._
 ---
 
+## Python 2.7 is now discontinued and gtk has always been a pain for cross-platform use, so I am in the process of writing a new interface in Python 3 and Qt that I hope will be working before too long.
+
 This is the documentation for the DStat interface software.
 The DStat interface is written primarily in Python and runs on Linux, Mac, and Windows.
 It is the main method for running experiments on the DStat, controlling experimental parameters and collecting and plotting data.
@@ -14,6 +16,7 @@ It currently has no abilities for analyzing recorded data or opening previously
 	    2.  [Old Homebrew Instructions](#old-homebrew-instructions)
     1.  [Linux](#linux)
     2.  [Windows](#windows)
+    3.  [Upgrading](#upgrading)
 2. [Getting Started](#Getting-Started)
 
 # Introduction
@@ -30,8 +33,10 @@ dstat-interface has moved to gtk+3 and we now recommend Anaconda/Miniconda for i
 
   1. [Install Miniconda](https://repo.continuum.io/miniconda/Miniconda2-latest-MacOSX-x86_64.sh) It doesn't matter if you pick Python 2.7 or 3.5—this just sets Miniconda's default Python. (Skip if Miniconda or Anaconda are already installed)
   
-  2. Create a new environment for dstat. In a terminal type:
-````conda create -n dstat -c mdryden python=2.7 dstat-interface dstat-interface-deps````
+  2. Download the [conda env file](conda-env.yml).
+  
+  3. In the terminal, create the dstat environment (replacing </path/to/conda-env.yml> with the actual path to the file on your computer):
+````conda env create -f </path/to/conda-env.yml>````
 
   3. Then to run dstat-interface:
     ````source activate dstat
@@ -40,10 +45,12 @@ dstat-interface has moved to gtk+3 and we now recommend Anaconda/Miniconda for i
 #### Old Homebrew Instructions
 The easiest way to get most of the necessary requirements to run dstat-interface is using [Homebrew](http://brew.sh):
 
-    brew tap homebrew/python
-    brew update
-    brew install python gobject-introspection gtk+3 pygobject3 py2cairo scipy zeromq 
-    brew install matplotlib --with-pygtk
+```shell
+brew tap homebrew/python
+brew update
+brew install python gobject-introspection gtk+3 pygobject3 py2cairo scipy zeromq 
+brew install matplotlib --with-pygtk
+```
 
 Be patient on the last step—matplotlib needs to be compiled and may take 2 or 3 minutes.
 
@@ -56,17 +63,26 @@ The final requirements, can be installed using python's pip system:
 ## Linux
 Linux prerequisite installation is similar to that of MacOS with Homebrew, only using your distribution's native package manager rather than Homebrew, and X11 will likely be installed already. Some distributions may not have packages available for installing matplotlib or numpy, in which case, they should be installed using pip.
 
-The final requirements, can be installed using python's pip system:
+These instructions were tested on Ubuntu 17.04:
 
-    pip install pandas pyserial pyzmq pyyaml seaborn zmq-plugin
+````shell
+sudo apt-get install gobject-introspection python-gobject python-pip
+pip install dstat-interface
+````
+
+You will need to add your user to the `dialout` group to access virtual serial ports (replace <user> with your username):
+
+```shell
+sudo usermod -a -G dialout <user>
+```
 
 ## Windows
 The following terminal commands will result in a full installation of dstat-interface and its requirements, assuming [64-bit Miniconda][1] is installed:
 
-````
+```shell
 conda create -n dstat -c mdryden python=2 dstat-interface
 activate dstat
-````
+```
 
 To finish the installation, GTK+3 and its Python bindings must be installed:
 
@@ -84,12 +100,35 @@ and `deactivate` will return to the root environment.
 
 Therefore, to run dstat-interface from our environment, we must first activate it (if not already done) before launching it:
 
-    activate dstat
-    python -m dstat_interface.main
+```shell
+activate dstat
+python -m dstat_interface.main
+```
 
 [1]: https://repo.continuum.io/miniconda/Miniconda2-latest-Windows-x86_64.exe
 [2]: https://sourceforge.net/projects/pygobjectwin32/
 
+## Upgrading
+
+Anaconda builds can be upgraded to the latest version by issuing this command (from an activated conda environment):
+
+```shell
+conda upgrade -c mdryden dstat-interface  # For MacOS, be sure to upgrade dstat-interface-deps as well
+```
+
+pip installs can be upgraded similarly:
+
+```shell
+pip install --upgrade dstat-interface
+```
+
+You can also run development builds directly from a cloned git repository (from an activated conda environment):
+
+```shell
+cd ~/src/dstat-interface/dstat_interface # Replace with path to dstat_interface folder inside repository
+python -m main
+```
+
 # Getting started
 ## Interface overview
 

conda-env.yml

+name: dstat
+channels:
+  - conda-forge
+  - mdryden
+  - defaults
+dependencies:
+  - ca-certificates=2018.4.16=0
+  - certifi=2018.4.16=py27_0
+  - nb_conda_kernels=2.1.0=py27_0
+  - openssl=1.0.2o=0
+  - appnope=0.1.0=py27_0
+  - backports=1.0=py27_0
+  - backports_abc=0.5=py27_0
+  - bleach=1.5.0=py27_0
+  - configparser=3.5.0=py27_0
+  - curl=7.54.1=0
+  - cycler=0.10.0=py27_0
+  - decorator=4.1.2=py27_0
+  - entrypoints=0.2.3=py27_0
+  - enum34=1.1.6=py27_0
+  - expat=2.1.0=0
+  - freetype=2.5.5=2
+  - funcsigs=1.0.2=py27hb9f6266_0
+  - functools32=3.2.3.2=py27_0
+  - get_terminal_size=1.0.0=py27_0
+  - gettext=0.19.8=1
+  - git=2.11.1=0
+  - html5lib=0.9999999=py27_0
+  - icu=54.1=0
+  - intel-openmp=2018.0.0=h8158457_8
+  - ipykernel=4.6.1=py27_0
+  - ipython=5.3.0=py27_0
+  - ipython-notebook=4.0.4=py27_0
+  - ipython_genutils=0.2.0=py27_0
+  - jbig=2.1=0
+  - jinja2=2.9.6=py27_0
+  - jpeg=9b=0
+  - jsonschema=2.6.0=py27_0
+  - jupyter_client=5.1.0=py27_0
+  - jupyter_core=4.3.0=py27_0
+  - krb5=1.13.2=0
+  - libcxx=4.0.1=h579ed51_0
+  - libcxxabi=4.0.1=hebd6815_0
+  - libffi=3.2.1=1
+  - libgfortran=3.0.1=h93005f0_2
+  - libiconv=1.14=0
+  - libpng=1.6.30=1
+  - libssh2=1.8.0=0
+  - libtiff=4.0.6=3
+  - llvmlite=0.21.0=py27hac8ee23_0
+  - markupsafe=1.0=py27_0
+  - matplotlib=2.0.2=np113py27_0
+  - mistune=0.7.4=py27_0
+  - mkl=2018.0.1=hfbd8650_4
+  - nbconvert=5.2.1=py27_0
+  - nbformat=4.4.0=py27_0
+  - notebook=5.0.0=py27_0
+  - numba=0.36.2=np113py27h7c931aa_0
+  - numpy=1.13.3=py27h62f9060_0
+  - pandas=0.20.3=py27_0
+  - pandocfilters=1.4.2=py27_0
+  - path.py=10.3.1=py27_0
+  - pathlib2=2.3.0=py27_0
+  - patsy=0.4.1=py27_0
+  - pcre=8.39=1
+  - pexpect=4.2.1=py27_0
+  - pickleshare=0.7.4=py27_0
+  - pip=9.0.1=py27_1
+  - prompt_toolkit=1.0.15=py27_0
+  - ptyprocess=0.5.2=py27_0
+  - pygments=2.2.0=py27_0
+  - pyparsing=2.2.0=py27_0
+  - pyqt=5.6.0=py27_2
+  - python=2.7.13=0
+  - python-dateutil=2.6.1=py27_0
+  - pytz=2017.2=py27_0
+  - pyyaml=3.12=py27_0
+  - pyzmq=16.0.2=py27_0
+  - qt=5.6.2=2
+  - readline=6.2=2
+  - scandir=1.5=py27_0
+  - scipy=1.0.0=py27h793f721_0
+  - seaborn=0.8=py27_0
+  - setuptools=36.4.0=py27_0
+  - simplegeneric=0.8.1=py27_1
+  - singledispatch=3.4.0.3=py27_0
+  - sip=4.18=py27_0
+  - six=1.10.0=py27_0
+  - sqlite=3.13.0=0
+  - ssl_match_hostname=3.5.0.1=py27_0
+  - statsmodels=0.8.0=np113py27_0
+  - subprocess32=3.2.7=py27_0
+  - terminado=0.6=py27_0
+  - testpath=0.3.1=py27_0
+  - tk=8.5.18=0
+  - tornado=4.5.2=py27_0
+  - traitlets=4.3.2=py27_0
+  - wcwidth=0.1.7=py27_0
+  - wheel=0.29.0=py27_0
+  - xz=5.2.3=0
+  - yaml=0.1.6=0
+  - zlib=1.2.11=0
+  - adwaita-icon-theme=3.24.0=1
+  - arrow=0.10.0=py27_0
+  - at-spi2-atk=2.24.1=2
+  - at-spi2-core=2.24.1=2
+  - atk=2.24.0=3
+  - cairo-gobject=1.14.8=8
+  - dbus-client=1.10.18=0
+  - dfu-programmer=0.7.2=2
+  - dstat-interface=1.4.6=py27_0
+  - dstat-interface-deps=1.0=0
+  - gdk-pixbuf=2.36.6=2
+  - glib=2.52.2=5
+  - gobject-introspection=1.52.1=2
+  - gtk3=3.22.15=4
+  - harfbuzz=1.4.6=3
+  - libepoxy=1.4.2=5
+  - libusb=1.0.21=0
+  - pango=1.40.6=2
+  - pixman=0.34.0=1
+  - py2cairo=1.10.0=py27_0
+  - pygobject3=3.24.2=py27_3
+  - pyserial=3.3=py27_0
+  - zmq-plugin=0.2.post14=py27_0
+  - pip:
+    - backports.shutil-get-terminal-size==1.0.0
+    - backports.shutil-which==3.5.1
+    - backports.ssl-match-hostname==3.5.0.1
+    - chardet==3.0.4
+    - colorama==0.3.9
+    - idna==2.6
+    - paver==1.2.4
+    - pygobject==3.24.1
+    - requests==2.18.4
+    - urllib3==1.22
+    - vmprof==0.4.10

dstat_interface/analysis.py → dstat_interface/core/analysis.py

@@ -21,16 +21,19 @@
 Functions for analyzing data.
 """ 
 import logging
+import os
 
 from numpy import mean, trapz
 
-logger = logging.getLogger('dstat.analysis')
+logger = logging.getLogger(__name__)
+mod_dir = os.path.dirname(os.path.abspath(__file__))
 
 class AnalysisOptions(object):
     """Analysis options window."""
     def __init__(self, builder):
         self.builder = builder
-        self.builder.add_from_file('interface/analysis_options.glade')
+        self.builder.add_from_file(
+            os.path.join(mod_dir, 'interface/analysis_options.glade'))
         self.builder.connect_signals(self)
         
         self.window = self.builder.get_object('analysis_dialog')

dstat_interface/core/dstat/__init__.py

+from . import comm
+from . import dfu
+from . import state
\ No newline at end of file

dstat_interface/core/dstat/boards.py

+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#     DStat Interface - An interface for the open hardware DStat potentiostat
+#     Copyright (C) 2017  Michael D. M. Dryden - 
+#     Wheeler Microfluidics Laboratory <http://microfluidics.utoronto.ca>
+#         
+#     
+#     This program is free software: you can redistribute it and/or modify
+#     it under the terms of the GNU General Public License as published by
+#     the Free Software Foundation, either version 3 of the License, or
+#     (at your option) any later version.
+#     
+#     This program is distributed in the hope that it will be useful,
+#     but WITHOUT ANY WARRANTY; without even the implied warranty of
+#     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#     GNU General Public License for more details.
+#     
+#     You should have received a copy of the GNU General Public License
+#     along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from __future__ import division, absolute_import, print_function, unicode_literals
+import sys
+import inspect
+import logging
+
+from pkg_resources import parse_version, parse_requirements
+
+logger = logging.getLogger(__name__)
+
+
+class BaseBoard(object):
+    pcb_version = 'x.x.x'
+    booster = False
+
+    def __init__(self):
+        self.max_freq = 5000
+        self.max_scans = 255
+        self.max_time = 65535
+
+        self.setup()
+        assert len(self.gain) == self.gain_settings
+        assert len(self.gain_labels) == self.gain_settings
+        if self.gain_trim is not None:
+            assert len(self.gain_trim) == self.gain_settings
+
+    def setup(self):
+        """Override in subclasses to provide correct numbers"""
+        self.gain = [1, 1e2, 3e3, 3e4, 3e5, 3e6, 3e7, 1e8]
+        self.gain_labels = ["Bypass", "100 Ω (15 mA FS)", "3 kΩ (500 µA FS)",
+                            "30 kΩ (50 µA FS)", "300 kΩ (5 µA FS)",
+                            "3 MΩ (500 nA FS)", "30 MΩ (50 nA FS)",
+                            "100 MΩ (15 nA FS)"
+                           ]
+        self.gain_trim = [None, 'r100_trim', 'r3k_trim',
+                          'r30k_trim', 'r300k_trim', 'r3M_trim',
+                          'r30M_trim', 'r100M_trim']
+        self.gain_settings = len(self.gain)
+        self.gain_default_index = 2
+        self.re_voltage_scale = 1
+
+    def test_mv(self, mv):
+        """Return true if voltage in mV is in range."""
+        dac = float(mv)*self.re_voltage_scale/(3000./65536) + 32768
+
+        if 0 <= dac <= 65535:
+            return True
+        else:
+            return False
+
+    def test_freq(self, hz):
+        """Return true if frequency in Hz is in range."""
+        return 0 < float(hz) < self.max_freq
+
+    def test_scans(self, n):
+        """Return true if number of scans is valid."""
+        return 0 < int(n) < self.max_scans
+
+    def test_s(self, s):
+        """Return true if time in integer seconds is valid."""
+        return 0 < int(s) < self.max_time
+
+
+class V1_1Board(BaseBoard):
+    pcb_version = '1.1'
+
+    def setup(self):
+        self.gain = [1e2, 3e2, 3e3, 3e4, 3e5, 3e6, 3e7, 5e8]
+        self.gain_labels = [None, "300 Ω (5 mA FS)",
+                            "3 kΩ (500 µA FS)", "30 kΩ (50 µA FS)",
+                            "300 kΩ (5 µA FS)", "3 MΩ (500 nA FS)",
+                            "30 MΩ (50 nA FS)", "500 MΩ (3 nA FS)"
+                            ]
+        self.gain_trim = None
+        self.gain_settings = len(self.gain)
+        self.gain_default_index = 2
+        self.re_voltage_scale = 1
+
+
+class V1_2Board(BaseBoard):
+    pcb_version = '1.2'
+
+    def setup(self):
+        self.gain = [1, 1e2, 3e3, 3e4, 3e5, 3e6, 3e7, 1e8]
+        self.gain_labels = ["Bypass", "100 Ω (15 mA FS)", "3 kΩ (500 µA FS)",
+                            "30 kΩ (50 µA FS)", "300 kΩ (5 µA FS)",
+                            "3 MΩ (500 nA FS)", "30 MΩ (50 nA FS)",
+                            "100 MΩ (15 nA FS)"
+                           ]
+        self.gain_trim = [None, 'r100_trim', 'r3k_trim',
+                          'r30k_trim', 'r300k_trim', 'r3M_trim',
+                          'r30M_trim', 'r100M_trim']
+        self.gain_settings = len(self.gain)
+        self.gain_default_index = 2
+        self.re_voltage_scale = 1
+
+
+def __get_all_subclasses(cls):
+    all_subclasses = []
+
+    for subclass in cls.__subclasses__():
+        all_subclasses.append(subclass)
+        all_subclasses.extend(__get_all_subclasses(subclass))
+
+    return all_subclasses
+
+
+def find_board(version, booster=False):
+    """Returns highest compatible board class or None if none available."""
+    boards = __get_all_subclasses(BaseBoard)
+    candidates = []
+    for board in boards:
+        req = next(parse_requirements('dstat~={}'.format(board.pcb_version)))
+        if board.booster == booster and version in req:
+            candidates.append(board)
+    try:
+        picked = sorted(candidates,
+                        key=lambda board: parse_version(board.pcb_version))[-1]
+        logger.info("Picked %s", picked)
+        return picked
+    except IndexError:
+        logger.warning("No matching board definition for ver: %s.", version)
+        return None

dstat_interface/core/dstat/dfu.py

+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#     DStat Interface - An interface for the open hardware DStat potentiostat
+#     Copyright (C) 2014  Michael D. M. Dryden -
+#     Wheeler Microfluidics Laboratory <http://microfluidics.utoronto.ca>
+#
+#
+#     This program is free software: you can redistribute it and/or modify
+#     it under the terms of the GNU General Public License as published by
+#     the Free Software Foundation, either version 3 of the License, or
+#     (at your option) any later version.
+#
+#     This program is distributed in the hope that it will be useful,
+#     but WITHOUT ANY WARRANTY; without even the implied warranty of
+#     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#     GNU General Public License for more details.
+#
+#     You should have received a copy of the GNU General Public License
+#     along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from __future__ import (absolute_import, division,
+                        print_function, unicode_literals)
+
+import subprocess
+import sys
+import os
+import time
+import logging
+from tempfile import mkdtemp
+from zipfile import ZipFile
+
+if sys.version_info >= (3,):
+    import urllib.request as urllib2
+    import urllib.parse as urlparse
+else:
+    import urllib2
+    import urlparse
+
+logger = logging.getLogger(__name__)
+
+try:
+    import gi
+    gi.require_version('Gtk', '3.0')
+    from gi.repository import Gtk
+except ImportError:
+    print("ERR: GTK not available")
+    sys.exit(1)
+
+import serial
+
+from . import state
+from .comm import dstat_logger, exp_logger
+
+fwurl = "http://microfluidics.utoronto.ca/gitlab/api/v4/projects/4/jobs/artifacts/master/download?job=1.2.3&private_token=zkgSx1FaaTP7yLyFKkX6"
+
+
+class FWDialog(object):
+    def __init__(self, parent, connect, stop_callback, disconnect_callback, signal='activate'):
+        self.parent = parent
+        self.stop = stop_callback
+        self.disconnect = disconnect_callback
+        connect.connect(signal, self.activate)
+    
+    def activate(self, widget=None, data=None):
+        for name, result in assert_deps().items():
+            if result is not True:
+                logger.error("Can't use firmware update module.")
+                self.missing_deps()
+                return
+                
+        self.stop()  # Stop OCP
+        version_result, master = test_firmware_version() 
+        
+        if version_result is False:
+            self.git_error()
+            return
+        
+        if version_result == 'latest':
+            message = "Your firmware is already up to date. (or on a descendent branch)"
+            secondary = "Click yes to reflash firmware anyways."
+        elif version_result == 'devel':
+            message = "Your firmware is not on the master branch."
+            secondary = "You may have a development version. " \
+                        "Click yes to reflash firmware anyways."
+        elif version_result == 'old':
+            message = "Your firmware is out of date."
+            secondary = "Click yes to flash the latest firmware."
+        
+        dialog = Gtk.MessageDialog(self.parent, 0, Gtk.MessageType.INFO,
+                                   Gtk.ButtonsType.YES_NO, message)
+        dialog.format_secondary_text(secondary)
+        dialog.get_content_area().add(
+            Gtk.Label(
+                label="Installed version: {}".format(state.firmware_version)))
+                
+        dialog.get_content_area().add(
+            Gtk.Label(label="Latest version: {}".format(master)))
+                
+        dialog.show_all()
+        response = dialog.run()
+        
+        if response == Gtk.ResponseType.YES:
+            try:
+                download_fw()
+            except:
+                self.dl_error()
+                return
+            
+            dstat_enter_dfu()
+            
+            self.dfu_notice()
+            self.disconnect()
+            try:
+                dfu_program()
+            except:
+                self.dfu_error()
+                
+            dialog.destroy()
+            
+        else:
+            dialog.destroy()
+    
+    def missing_deps(self):
+        dialog = Gtk.MessageDialog(
+                     self.parent, 0, Gtk.MessageType.ERROR,
+                     Gtk.ButtonsType.OK, "Missing Dependencies")
+        
+        dialog.format_secondary_text('Check console for more info.')
+        
+        dialog.connect('response', self.destroy)
+        dialog.show()
+    
+    def git_error(self):
+        dialog = Gtk.MessageDialog(
+                     self.parent, 0, Gtk.MessageType.ERROR,
+                     Gtk.ButtonsType.OK, "Git Error")
+        
+        dialog.format_secondary_text('Check console for more info.')
+        
+        dialog.connect('response', self.destroy)
+        dialog.show()
+        
+    def dl_error(self):
+        dialog = Gtk.MessageDialog(
+                     self.parent, 0, Gtk.MessageType.ERROR,
+                     Gtk.ButtonsType.OK, "Download Error")
+        
+        dialog.format_secondary_text('Check console for more info.')
+        
+        dialog.connect('response', self.destroy)
+        dialog.show()
+    
+    def dfu_notice(self):
+        dialog = Gtk.MessageDialog(
+                     self.parent, 0, Gtk.MessageType.INFO,
+                     Gtk.ButtonsType.OK, "Note about DFU")
+        
+        dialog.format_secondary_text("Click OK once the DStat has connected in "
+            + "DFU mode. Windows doesn't seem to like the automatic reboot. " 
+            + "Try holding down the reset button while plugging the "
+            + 'USB port in (No LEDs should be lit), then click OK. Make sure '
+            + 'the DFU driver from the dfu-programmer directory is installed.')
+        
+        dialog.run()
+        dialog.destroy()
+        
+    def dfu_error(self):
+        dialog = Gtk.MessageDialog(
+                     self.parent, 0, Gtk.MessageType.ERROR,
+                     Gtk.ButtonsType.OK, "Could not update over DFU")
+        
+        dialog.format_secondary_text('Check console for more info.')
+        
+        dialog.connect('response', self.destroy)
+        dialog.show()
+        
+    def destroy(self, widget=None, data=None):
+        widget.destroy()
+    
+
+def assert_deps():
+    deps = {'git' : 'git --version',
+            'dfu-programmer' : 'dfu-programmer --version'}
+    
+    result = {}
+    
+    for key, command in deps.items():        
+        try:
+            output = subprocess.check_output(command.split(),
+                                             stderr=subprocess.STDOUT).decode(sys.stdout.encoding)
+            logger.info("Assert: %s", command)
+            logger.info("Result: %s", output)
+            result[key] = True
+        except subprocess.CalledProcessError:
+            logger.warning("{} is not available.".format(key))
+            result[key] = False
+            
+    return result
+
+
+def download_fw():  # from https://stackoverflow.com/a/16518224
+    temp_dir = mkdtemp()
+    logger.info("Temporary directory: {}".format(temp_dir))
+    os.chdir(temp_dir)  # Go to temporary directory
+    
+    u = urllib2.urlopen(fwurl)
+
+    scheme, netloc, path, query, fragment = urlparse.urlsplit(fwurl)
+    filename = os.path.basename(path)
+    if not filename:
+        filename = 'downloaded.file'
+
+    with open(filename, 'wb') as f:
+        meta = u.info()
+        meta_func = meta.getheaders if hasattr(meta, 'getheaders') else meta.get_all
+        meta_length = meta_func("Content-Length")
+        file_size = None
+        if meta_length:
+            file_size = int(meta_length[0])
+        logger.info("Downloading: {0} Bytes: {1}".format(fwurl, file_size))
+
+        file_size_dl = 0
+        block_sz = 8192
+        while True:
+            buffer = u.read(block_sz)
+            if not buffer:
+                break
+
+            file_size_dl += len(buffer)
+            f.write(buffer)
+
+            status = "{0:16}".format(file_size_dl)
+            if file_size:
+                status += "   [{0:6.2f}%]".format(file_size_dl * 100 / file_size)
+            status += chr(13)
+            logger.info(status)
+    
+    with ZipFile(filename, mode='r') as z:
+        fw_path = z.extract('dstat-firmware.hex')
+    
+    return fw_path
+
+
+def test_firmware_version(current=None):
+    if current is None:
+        current = state.firmware_version
+    
+    temp_dir = mkdtemp()
+    logger.info("Temporary directory: {}".format(temp_dir))
+    os.chdir(temp_dir)  # Go to temporary directory
+    
+    command = "git clone http://microfluidics.utoronto.ca/gitlab/dstat/dstat-firmware.git"
+    logger.info('Cloning master')
+    try:
+        output = subprocess.check_output(command.split(),
+                                         stderr=subprocess.STDOUT).decode(sys.stdout.encoding)
+    except subprocess.CalledProcessError as e:
+        logger.error("git failed with error code {}".format(e.returncode))
+        logger.error("Output: {}".format(e.output))
+        return False, None
+    logger.info(output)
+    
+    os.chdir("./dstat-firmware")
+    
+    command = "git rev-parse --short master"
+    master = subprocess.check_output(command.split(), stderr=subprocess.STDOUT).decode(sys.stdout.encoding)
+    logger.info("Current master commit: {}".format(master))
+    
+    command = "git merge-base --is-ancestor master {}".format(current)
+    test = subprocess.call(command.split())
+    logger.info(test)
+
+    if test == 0:  # already newest
+        logger.info('Firmware is latest available (or on a newer descendent branch).')
+        return 'latest', master
+    elif test == 1:  # old version
+        logger.info('Firmware is out of date.')
+        return 'old', master
+    elif test == 128:  # newer or different branch
+        logger.info('Firmware is not on the master branch.')
+        return 'devel', master
+    else:
+        logger.error('Unexpected git error. Git exited {}'.format(test))
+        return False, None
+
+
+def dfu_program(path='./dstat-firmware.hex'):
+    """Tries to program DStat over USB with DFU with hex file at path."""
+    try:
+        command = "dfu-programmer atxmega256a3u erase"
+        output = subprocess.check_output(command.split(),
+                                          stderr=subprocess.STDOUT).decode(sys.stdout.encoding)
+        logger.info("%s\n%s", command, output)
+        command = "dfu-programmer atxmega256a3u flash {}".format(path)
+        output = subprocess.check_output(command.split(),
+                                          stderr=subprocess.STDOUT).decode(sys.stdout.encoding)
+        logger.info("%s\n%s", command, output)
+        command = "dfu-programmer atxmega256a3u launch"
+        output = subprocess.check_output(command.split(),
+                                          stderr=subprocess.STDOUT).decode(sys.stdout.encoding)
+        logger.info("%s\n%s", command, output)
+    except subprocess.CalledProcessError as e:
+        logger.error("{} failed with output:".format(" ".join(e.cmd)))
+        logger.error(e.output)
+        raise
+
+
+def dstat_enter_dfu():
+    """Tries to contact DStat and get version. Stores version in state.
+    If no response, returns False, otherwise True.
+        
+    Arguments:
+    ser_port -- address of serial port to use
+    """
+    exp = DFUMode()
+    state.ser.start_exp(exp)
+    while True:
+        result = state.ser.get_proc(block=True)
+        if result in ('SERIAL_ERROR', 'DONE'):
+            break
+    logger.info(result)
+
+    time.sleep(.1)
+    
+    return True
+
+
+class DFUMode(object):
+    def __init__(self):
+        pass
+        
+    def run(self, ser, ctrl_pipe, data_pipe):
+        """Tries to contact DStat and get version. Returns a tuple of
+        (major, minor). If no response, returns empty tuple.
+            
+        Arguments:
+        ser_port -- address of serial port to use
+        """
+        status = None
+        try:
+            ser.write(b'!2\n')
+            exp_logger.info('!2')
+    
+            for i in range(10):
+                if ser.readline().rstrip() == b"@ACK 2":
+                    dstat_logger.info('@ACK 2')
+                    ser.write(b'SF\n')
+                    exp_logger.info('SF')
+                    status = "DONE"
+                    time.sleep(5)
+                    break
+                else:
+                    time.sleep(.5)
+                    ser.reset_input_buffer()
+                    ser.write(b'!2\n')
+                    exp_logger.info('!2')
+                    time.sleep(.1)
+
+        except UnboundLocalError as e:
+            status = "SERIAL_ERROR"
+        except serial.SerialException as e:
+            logger.error('SerialException: %s', e)
+            status = "SERIAL_ERROR"
+        finally:
+            return status
+
+
+if __name__ == "__main__":
+    log_handler = logging.StreamHandler()
+    log_formatter = logging.Formatter(
+                        fmt='%(asctime)s %(levelname)s: [%(name)s] %(message)s',
+                        datefmt='%H:%M:%S'
+                    )
+    log_handler.setFormatter(log_formatter)
+    logger.setLevel(level=logging.INFO)
+    logger.addHandler(log_handler)
+        
+    dstat_enter_dfu()
+    time.sleep(2)
+    dfu_program(sys.argv[1])
\ No newline at end of file

dstat_interface/core/dstat/state.py

+from collections import OrderedDict
+
+
+def reset():
+    """Resets state variables."""
+
+    settings = OrderedDict()
+    ser = None
+    dstat_version = None
+    firmware_version = None
+    board_instance = None
+
+
+def get_state():
+    """
+    Get state variables
+    :return dict of pickleable state variables
+    """
+    return {'settings': settings,
+            'dstat_version': dstat_version,
+            'firmware_version': firmware_version,
+            'board_instance': board_instance}
+
+
+settings = OrderedDict()
+ser = None
+dstat_version = None
+firmware_version = None
+board_instance = None

dstat_interface/core/experiments/__init__.py

+# __all__ = []
+#
+# import pkgutil
+# import inspect
+# from . import cal, chronoamp, cv
+#
+#
+# for loader, name, is_pkg in pkgutil.walk_packages(__path__):
+#     print loader, name, is_pkg
+#     module = loader.find_module(name).load_module(name)
+#
+#     for name, value in inspect.getmembers(module):
+#         if name.startswith('__'):
+#             continue
+#
+#         globals()[name] = value
+#         __all__.append(name)
\ No newline at end of file

dstat_interface/experiments/cal.py → dstat_interface/core/experiments/cal.py

@@ -21,15 +21,14 @@
 import time
 import struct
 import logging
-
-from errors import InputError, VarError
+logger = logging.getLogger(__name__)
 
 import serial
 
-logger = logging.getLogger("dstat.experiments.cal")
+from ..errors import InputError, VarError
+from ..dstat import state
+from ..experiments.experiment_template import Experiment, dstat_logger
 
-import state
-from experiments.experiment_template import Experiment, dstat_logger
 
 def measure_offset(time):
     gain_trim_table = [None, 'r100_trim', 'r3k_trim', 'r30k_trim', 'r300k_trim',
@@ -48,6 +47,7 @@ def measure_offset(time):
         
     return gain_offset
 
+
 class CALExp(Experiment):
     id = 'cal'
     """Offset calibration experiment"""
@@ -57,15 +57,16 @@ class CALExp(Experiment):
         self.scan = 0
         self.data = []
 
-        self.commands = ["EA2 3 1 ", "EG", "ER"]
+        self.commands = ["EA2 3 1 ", "EG"]
 
         self.commands[1] += str(self.parameters['gain'])
         self.commands[1] += " "
         self.commands[1] += "0 "
-        self.commands[2] += "1 32768 "
-        self.commands[2] += str(self.parameters['time'])
-        self.commands[2] += " "
-        self.commands[2] += "0 " # disable photodiode interlock
+
+        self.commands.append(
+            ("ER1 0", ["32768", str(self.parameters['time'])])
+        )
+
         
     def serial_handler(self):
         """Handles incoming serial transmissions from DStat. Returns False
@@ -100,6 +101,7 @@ class CALExp(Experiment):
                     elif line.lstrip().startswith("@DONE"):
                         dstat_logger.debug(line.lstrip().rstrip())
                         self.serial.flushInput()
+                        self.experiment_done()
                         return True
                         
         except serial.SerialException:
@@ -114,13 +116,16 @@ class CALExp(Experiment):
         seconds, milliseconds, current = struct.unpack('<HHl', data)
         return current
     
-    def data_postprocessing(self):
+    def experiment_done(self):
         """Averages data points
         """
-        
-        sum = 0
-        self.data[0] = 0 # Skip first point
-        
+        try:
+            sum = 0
+            self.data[0] = 0 # Skip first point
+
+        except IndexError:
+            return
+
         for i in self.data:
             sum += i
         

dstat_interface/experiments/chronoamp.py → dstat_interface/core/experiments/chronoamp.py

@@ -2,47 +2,59 @@ import time
 import struct
 import numpy as np
 import serial
+from ..interface.plot import mean, plotSpectrum, findBounds
+
+from .experiment_template import PlotBox, Experiment, exp_logger
 
-from experiments.experiment_template import PlotBox, Experiment, exp_logger
 
 class ChronoampBox(PlotBox):
+    def setup(self):
+        self.plot_format = {
+            'current_time': {'xlabel': "Time (s)",
+                             'ylabel': "Current (A)"
+                            }
+        }
+
     def format_plots(self):
         """
         Creates and formats subplots needed. Overrides superclass.
         """
-        self.subplots = {'current_time' : self.figure.add_subplot(111)}
+        self.subplots = {'current_time': self.figure.add_subplot(111)}
         
         for key, subplot in self.subplots.items():
             subplot.ticklabel_format(style='sci', scilimits=(0, 3),
                                      useOffset=False, axis='y')
             subplot.plot([],[])
-        
+            subplot.set_xlabel(self.plot_format[key]['xlabel'])
+            subplot.set_ylabel(self.plot_format[key]['ylabel'])
+
+
 class Chronoamp(Experiment):
     id = 'cae'
     """Chronoamperometry experiment"""
     def setup(self):
-        self.plots.append(ChronoampBox('current_time'))
-        
         self.datatype = "linearData"
         self.datalength = 2
         self.databytes = 8
         self.data = {'current_time' : [([],[])]}
         self.columns = ['Time (s)', 'Current (A)']
-        self.plot_format = {
-                            'current_time' : {
-                                'labels' : self.columns,
-                                'xlims' : (0, sum(self.parameters['time']))
-                                }
-                            }
+        self.total_time = sum(self.parameters['time'])
+        self.plotlims = {'current_time': {'xlims': (0, self.total_time)}
+                         }
         
         self.commands.append(
-            ("ER"+ str(len(self.parameters['potential'])) + " 0 ", [])
+            ("ER" + str(len(self.parameters['potential'])) + " 0 ", [])
             )
 
         for i in self.parameters['potential']:
             self.commands[-1][1].append(str(int(i*(65536./3000)+32768)))
         for i in self.parameters['time']:
             self.commands[-1][1].append(str(i))
+
+        plot = ChronoampBox(['current_time'])
+        plot.setlims('current_time', **self.plotlims['current_time'])
+
+        self.plots.append(plot)
       
     def data_handler(self, data_input):
         """Overrides Experiment method to not convert x axis to mV."""
@@ -65,7 +77,13 @@ class Chronoamp(Experiment):
 
         for i, item in enumerate(self.data['current_time'][line]):
             item.append(data[i])
-                
+
+    def get_progress(self):
+        try:
+            return self.data['current_time'][-1][0][-1]/self.total_time
+        except IndexError:
+            return 0
+
 class PDExp(Chronoamp):
     """Photodiode/PMT experiment"""
     id = 'pde'
@@ -83,29 +101,33 @@ class PDExp(Chronoamp):
                                 'xlims' : (0, int(self.parameters['time']))
                                 }
                             }
-        
+        self.total_time = int(self.parameters['time'])
+
         if self.parameters['shutter_true']:
             if self.parameters['sync_true']:
                 self.commands.append("EZ")
                 self.commands[-1] += str(self.parameters['sync_freq'])
                 self.commands[-1] += " "
             else:
-                self.commands.append("E2")        
-        
-        self.commands.append(("ER1 ", []))
-        
+                self.commands.append(("E2", []))
+
+        command = "ER1 "
+        params = []
+
         if self.parameters['interlock_true']:
-            self.commands[-1][0] += "1 "
+            command += "1 "
         else:
-            self.commands[-1][0] += "0 "
+            command += "0 "
             
         if self.parameters['voltage'] == 0: # Special case where V=0
-            self.commands[-1][1].append("65535")
+            params.append("65535")
         else:
-            self.commands[-1][1].append(str(int(
+            params.append(str(int(
                             65535-(self.parameters['voltage']*(65536./3000))))
                             )
-        self.commands[-1][1].append(str(self.parameters['time']))
+        params.append(str(self.parameters['time']))
+
+        self.commands.append((command, params))
 
         if self.parameters['shutter_true']:
             if self.parameters['sync_true']:

dstat_interface/experiments/cv.py → dstat_interface/core/experiments/cv.py

 import time
 import struct
 
-from experiments.experiment_template import PlotBox, Experiment
+from .experiment_template import PlotBox, Experiment
+
 
 class CVExp(Experiment):
     id = 'cve'
     """Cyclic Voltammetry experiment"""
     def setup(self):
+        self.plotlims['current_voltage']['xlims'] = tuple(
+                sorted((int(self.parameters['v1']), int(self.parameters['v2'])))
+                )
         super(CVExp, self).setup()
+
         self.datatype = "CVData"
         self.xlabel = "Voltage (mV)"
         self.ylabel = "Current (A)"
         self.datalength = 2 * self.parameters['scans']  # x and y for each scan
         self.databytes = 6  # uint16 + int32
-        self.plot_format['current_voltage']['xlims'] = (
-            int(self.parameters['v1']),
-            int(self.parameters['v2'])
-            )
         
         self.commands += "E"
         self.commands[2] += "C"
@@ -24,19 +25,44 @@ class CVExp(Experiment):
         self.commands[2] += " "
         self.commands[2] += str(self.parameters['dep_s'])
         self.commands[2] += " "
-        self.commands[2] += str(int(int(self.parameters['clean_mV'])*
-                                (65536./3000)+32768))
+        self.commands[2] += str(int(
+                                    int(self.parameters['clean_mV'])/
+                                    self.re_voltage_scale*
+                                    (65536./3000)+32768
+                                ))
         self.commands[2] += " "
-        self.commands[2] += str(int(int(self.parameters['dep_mV'])*
-                                (65536./3000)+32768))
+        self.commands[2] += str(int(
+                                    int(self.parameters['dep_mV'])/
+                                    self.re_voltage_scale*
+                                    (65536./3000)+32768
+                                ))
         self.commands[2] += " "
-        self.commands[2] += str(self.parameters['v1'])
+        self.commands[2] += str(int(
+                                    int(self.parameters['v1'])/
+                                    self.re_voltage_scale*
+                                    (65536./3000)+32768
+                                ))
         self.commands[2] += " "
-        self.commands[2] += str(self.parameters['v2'])
+        self.commands[2] += str(int(
+                                    int(self.parameters['v2'])/
+                                    self.re_voltage_scale*
+                                    (65536./3000)+32768
+                                ))
         self.commands[2] += " "
-        self.commands[2] += str(self.parameters['start'])
+        self.commands[2] += str(int(
+                                    int(self.parameters['start'])/
+                                    self.re_voltage_scale*
+                                    (65536./3000)+32768
+                                ))
         self.commands[2] += " "
         self.commands[2] += str(self.parameters['scans'])
         self.commands[2] += " "
-        self.commands[2] += str(self.parameters['slope'])
-        self.commands[2] += " "
\ No newline at end of file
+        self.commands[2] += str(int(
+                                    int(self.parameters['slope'])/
+                                    self.re_voltage_scale*
+                                    (65536./3000)
+                                ))
+        self.commands[2] += " "
+
+    def get_progress(self):
+        return (len(self.data['current_voltage'])-1) / float(self.parameters['scans'])