Developer documentation¶

The details printed in debug mode (requests and responses) are very useful for using the included dummy_serial port for unit testing purposes. For examples, see the file test/test_minimalmodbus.py.

Design considerations¶

My take on the design is that is should be as simple as possible, hence the name MinimalModbus, but it should implement the smallest number of functions needed for it to be useful. The target audience for this driver simply wants to talk to Modbus clients using a serial interface using some simple driver.

Only a few functions are implemented. It is very easy to implement lots of (seldom used) functions, resulting in buggy code with large fractions of it almost never used. It is instead much better to implement the features when needed/requested. There are many Modbus function codes, but I guess that most are not used.

There should be unittests for all functions, and mock communication data.

Errors should be caught as early as possible, and the error messages should be informative. For this reason there is type checking for for the parameters in most functions. This is rather un-pythonic, but is intended to give more clear error messages (for easier remote support).

Note that the term ‘address’ is ambiguous, why it is better to use the terms ‘register address’ or ‘slave address’.

Use only external links in the README.rst, otherwise they will not work on Python Package Index (PyPI). No Sphinx-specific constructs are allowed in that file.

Design priorities:

Easy to use

Catch errors early

Informative error messages

Good unittest coverage

General driver structure¶

The general structure of the program is shown here:

Function	Description
`read_register()`	One of the facades for `_generic_command()`.
`_generic_command()`	Generates payload, then calls `_perform_command()`.
`_perform_command()`	Embeds payload into error-checking codes etc, then calls `_communicate()`.
`_communicate()`	Handles raw bytes for communication via pySerial.

Most of the logic is located in separate (easy to test) functions on module level. For a description of them, see Internal documentation for MinimalModbus.

Number conversion to and from bytes¶

The Python module struct is used for conversion. See https://docs.python.org/3/library/struct.html

Several wrapper functions are defined for easy use of the conversion. These functions also do argument validity checking.

Data type	To bytes	From bytes
(internal usage)	`_num_to_two_bytes()`
Bit	`_bit_to_bytes()`	Same as for bits
Several bits	`_bits_to_bytes()`	`_bytes_to_bits()`
Integer (char, short)	`_num_to_two_bytes()`	`_two_bytes_to_num()`
Several registers	`_valuelist_to_bytes()`	`_bytes_to_valuelist()`
Long integer	`_long_to_bytes()`	`_bytes_to_long()`
Floating point number	`_float_to_bytes()`	`_bytes_to_float()`
String	`_textstring_to_bytes()`	`_bytes_to_textstring()`

For compability with both Python2 and Python3, earlier versions of this module did store data internally as bytestrings. Now that Python2 support has been dropped, the internal representation of data is using Python bytes objects.

Unit testing¶

Unit tests are provided in the tests subfolder. To run them:

make test

The unittests uses previosly recorded communication data for the testing.

A dummy/mock/stub for the serial port, dummy_serial, is provided for test purposes. See Documentation for dummy_serial (which is a serial port mock).

The test coverage analysis is found at https://codecov.io/github/pyhys/minimalmodbus?branch=master.

Hardware tests are performed using a Delta DTB4824 process controller together with a USB-to-RS485 converter. See Internal documentation for hardware testing of MinimalModbus using DTB4824 for more information.

Run it with:

python3 tests/test_deltaDTB4824.py

The baudrate, portname and mode can optionally be set from command line. For more details on testing with this hardware, see Internal documentation for hardware testing of MinimalModbus using DTB4824.

Making sure that error messages are informative for the user¶

To have a look on the error messages raised during unit testing of minimalmodbus, monkey-patch test_minimalmodbus.SHOW_ERROR_MESSAGES_FOR_ASSERTRAISES as seen here:

>>> import unittest
>>> import test_minimalmodbus
>>> test_minimalmodbus.SHOW_ERROR_MESSAGES_FOR_ASSERTRAISES = True
>>> suite = unittest.TestLoader().loadTestsFromModule(test_minimalmodbus)
>>> unittest.TextTestRunner(verbosity=2).run(suite)

This is part of the output:

testFunctioncodeNotInteger (test_minimalmodbus.TestEmbedPayload) ...
    TypeError('The functioncode must be an integer. Given: 1.0',)

    TypeError("The functioncode must be an integer. Given: '1'",)

    TypeError('The functioncode must be an integer. Given: [1]',)

    TypeError('The functioncode must be an integer. Given: None',)
ok
testKnownValues (test_minimalmodbus.TestEmbedPayload) ... ok
testPayloadNotString (test_minimalmodbus.TestEmbedPayload) ...
    TypeError('The payload should be a string. Given: 1',)

    TypeError('The payload should be a string. Given: 1.0',)

    TypeError("The payload should be a string. Given: ['ABC']",)

    TypeError('The payload should be a string. Given: None',)
ok
testSlaveaddressNotInteger (test_minimalmodbus.TestEmbedPayload) ...
    TypeError('The slaveaddress must be an integer. Given: 1.0',)

    TypeError("The slaveaddress must be an integer. Given: 'DEF'",)
ok
testWrongFunctioncodeValue (test_minimalmodbus.TestEmbedPayload) ...
    ValueError('The functioncode is too large: 222, but maximum value is 127.',)

    ValueError('The functioncode is too small: -1, but minimum value is 1.',)
ok
testWrongSlaveaddressValue (test_minimalmodbus.TestEmbedPayload) ...
    ValueError('The slaveaddress is too large: 248, but maximum value is 247.',)

    ValueError('The slaveaddress is too small: -1, but minimum value is 0.',)
ok

See test_minimalmodbus for details on how this is implemented.

It is possible to run just a few tests. To load a single class of test cases:

suite = unittest.TestLoader().loadTestsFromTestCase(test_minimalmodbus.TestSetBitOn)

If necessary:

reload(test_minimalmodbus.minimalmodbus)

Recording communication data for unittesting¶

With the known data output from an instrument, we can fine tune the inner details of the driver (code refactoring) without worrying that we change the output from the code. This data will be the ‘golden standard’ to which we test the code. Use as many as possible of the commands, and paste all the output in a text document. From this it is pretty easy to reshuffle it into unittest code.

Here is an example how to record communication data, which then is pasted into the test code (for use with a mock/dummy serial port). See for example Internal documentation for unit testing of MinimalModbus (click ‘[source]’ on right side, see RESPONSES at end of the page). Do like this:

>>> import minimalmodbus
>>> instrument_1 = minimalmodbus.Instrument('/dev/ttyUSB0',10)
>>> instrument_1.debug = True
>>> instrument_1.read_register(4097,1)
MinimalModbus debug mode. Writing to instrument: '\n\x03\x10\x01\x00\x01\xd0q'
MinimalModbus debug mode. Response from instrument: '\n\x03\x02\x07\xd0\x1e)'
200.0
>>> instrument_1.write_register(4097,325.8,1)
MinimalModbus debug mode. Writing to instrument: '\n\x10\x10\x01\x00\x01\x02\x0c\xbaA\xc3'
MinimalModbus debug mode. Response from instrument: '\n\x10\x10\x01\x00\x01U\xb2'
>>> instrument_1.read_register(4097,1)
MinimalModbus debug mode. Writing to instrument: '\n\x03\x10\x01\x00\x01\xd0q'
MinimalModbus debug mode. Response from instrument: '\n\x03\x02\x0c\xba\x996'
325.8
>>> instrument_1.read_bit(2068)
MinimalModbus debug mode. Writing to instrument: '\n\x02\x08\x14\x00\x01\xfa\xd5'
MinimalModbus debug mode. Response from instrument: '\n\x02\x01\x00\xa3\xac'
0
>>> instrument_1.write_bit(2068,1)
MinimalModbus debug mode. Writing to instrument: '\n\x05\x08\x14\xff\x00\xcf%'
MinimalModbus debug mode. Response from instrument: '\n\x05\x08\x14\xff\x00\xcf%'

This is also very useful for debugging drivers built on top of MinimalModbus.

Using the dummy serial port¶

A dummy serial port is included for testing purposes, see dummy_serial. Use it like this:

>>> import dummy_serial
>>> import test_minimalmodbus
>>> dummy_serial.RESPONSES = test_minimalmodbus.RESPONSES  # Load previously recorded responses
>>> import minimalmodbus
>>> minimalmodbus.serial.Serial = dummy_serial.Serial  # Monkey-patch a dummy serial port
>>> instrument = minimalmodbus.Instrument('DUMMYPORTNAME', 1)  # port name, slave address (in decimal)
>>> instrument.read_register(4097, 1)
823.6

In the example above there is recorded data available for read_register(4097, 1). If no recorded data is available, an error message is displayed:

>>> instrument.read_register(4098, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jonas/pythonprogrammering/minimalmodbus/trunk/minimalmodbus.py", line 174, in read_register
    return self._genericCommand(functioncode, registeraddress, numberOfDecimals=numberOfDecimals)
  File "/home/jonas/pythonprogrammering/minimalmodbus/trunk/minimalmodbus.py", line 261, in _genericCommand
    payloadFromSlave = self._performCommand(functioncode, payloadToSlave)
  File "/home/jonas/pythonprogrammering/minimalmodbus/trunk/minimalmodbus.py", line 317, in _performCommand
    response            = self._communicate(message)
  File "/home/jonas/pythonprogrammering/minimalmodbus/trunk/minimalmodbus.py", line 395, in _communicate
    raise IOError('No communication with the instrument (no answer)')
IOError: No communication with the instrument (no answer)

The dummy serial port can be used also with instrument drivers built on top of MinimalModbus:

>>> import dummy_serial
>>> import test_omegacn7500
>>> dummy_serial.RESPONSES = test_omegacn7500.RESPONSES  # Load previously recorded responses
>>> import omegacn7500
>>> omegacn7500.minimalmodbus.serial.Serial = dummy_serial.Serial  # Monkey-patch a dummy serial port
>>> instrument = omegacn7500.OmegaCN7500('DUMMYPORTNAME', 1)  # port name, slave address
>>> instrument.get_pv()
24.6

To see the generated request data (without bothering about the response):

>>> import dummy_serial
>>> import minimalmodbus
>>> minimalmodbus.serial.Serial = dummy_serial.Serial  # Monkey-patch a dummy serial port
>>> instrument = minimalmodbus.Instrument('DUMMYPORTNAME', 1)
>>> instrument.debug = True
>>> instrument.read_bit(2068)
MinimalModbus debug mode. Writing to instrument: '\x01\x02\x08\x14\x00\x01\xfb\xae'
MinimalModbus debug mode. Response from instrument: ''

(Then an error message appears)

Data encoding in Python2 and Python3¶

The string type has changed in Python3 compared to Python2. In Python3 the type bytes is used when communicating via pySerial.

Dependent on the Python version number, the data sent from MinimalModbus to pySerial has different types.

String constants¶

This is a string constant both in Python2 and Python3:

st = 'abc\x69\xe6\x03'

This is a bytes constant in Python3, but a string constant in Python2 (allowed for 2.6 and higher):

by = b'abc\x69\xe6\x03'

Type conversion in Python3¶

To convert a string to bytes, use one of these:

bytes(st, 'latin1')  # Note that 'ascii' encoding gives error for some values.
st.encode('latin1')

To convert bytes to string, use one of these:

str(by, encoding='latin1')
by.decode('latin1')

Encoding	Allowed range
ascii	0-127
latin-1	0-255

Corresponding in Python2¶

Ideally, we would like to use the same source code for Python2 and Python3. In Python 2.6 and higher there is the bytes() function for forward compatibility, but it is merely a synonym for str().

To convert from ‘bytes’(string) to string:

str(by)  # not possible to give encoding
by.decode('latin1')  # Gives unicode

To convert from string to ‘bytes’(string):

bytes(st)  # not possible to give encoding
st.encode('latin1')  # Can not be used for values larger than 127

It is thus not possible to use exactly the same code for both Python2 and Python3. Where it is unavoidable, use:

if sys.version_info[0] > 2:
    whatever

Extending MinimalModbus¶

It is straight-forward to extend MinimalModbus to handle more Modbus function codes. Use the method _perform_command() to send data to the slave, and to receive the response. Note that the API might change, as this is outside the official API.

This is easily tested in interactive mode. For example the method read_register() generates payload, which internally is sent to the instrument using _perform_command():

>>> instr.debug = True
>>> instr.read_register(5,1)
MinimalModbus debug mode. Writing to instrument: '\x01\x03\x00\x05\x00\x01\x94\x0b'
MinimalModbus debug mode. Response from instrument: '\x01\x03\x02\x00º9÷'
18.6

It is possible to use _perform_command() directly. You can use any Modbus function code (1-127), but you need to generate the payload yourself. Note that the same data is sent:

>>> instr._perform_command(3, b'\x00\x05\x00\x01')
MinimalModbus debug mode. Writing to instrument: '\x01\x03\x00\x05\x00\x01\x94\x0b'
MinimalModbus debug mode. Response from instrument: '\x01\x03\x02\x00º9÷'
'\x02\x00º'

Use this if you are to implement other Modbus function codes, as it takes care of CRC generation etc.

Other useful internal functions¶

There are several useful (module level) helper functions available in the minimalmodbus module. The module level helper functions can be used without any hardware connected. See Internal documentation for MinimalModbus. These can be handy when developing your own Modbus instrument hardware.

For example:

>>> minimalmodbus._calculate_crc(b'\x01\x03\x00\x05\x00\x01')
b'\x94\x0b'

And to embed the payload b'\x10\x11\x12' to slave address 1, with functioncode 16:

>>> minimalmodbus._embed_payload(1, MODE_RTU, 16, b'\x10\x11\x12')
b'\x01\x10\x10\x11\x12\x90\x98'

Playing with two’s complement:

>>> minimalmodbus._twos_complement(-1, bits=8)
255

Calculating the minimum silent interval (seconds) at a baudrate of 19200 bits/s:

>>> minimalmodbus._calculate_minimum_silent_period(19200)
0.0020052083333333332

Note that the API might change, as this is outside the official API.

Generate documentation¶

Use the top-level Make to generate HTML documentation:

make docs

Do linkchecking:

make linkcheck

Webpage¶

The HTML theme used is the Sphinx ‘sphinx_rtd_theme’ theme.

Codecov.io¶

Log in to https://codecov.io/ using your Github account.

Enable the webhook from GitHub to Codecov.io.

Notes on distribution¶

Installing the module from local files¶

In the top directory:

make install

To uninstall it:

make uninstall

How to generate a source distribution from the present development code¶

This will create a subfolder dist with the source in wheel format and in .tar.gz format:

make dist

Preparation for release¶

Change version number etc¶

Manually change the __version__ field in the minimalmodbus.py source file.
Manually change the release date in HISTORY.rst
Manually change the date and version in the CITATION.cff

(Note that the version number in the Sphinx configuration file doc/conf.py and in the file pyproject.toml are changed automatically. Also the copyright year in doc/conf.py is changed automatically).

How to number releases are described in PEP 440.

Code style checking etc¶

Automatically modify the formatting of the code:

make black

Check the code:

make pylint
make flake8

Check typing:

make mypy

Unittesting¶

Run unit tests:

make test

Show test coverage report:

make coverage

Also make tests using Delta DTB4824 hardware. See Internal documentation for hardware testing of MinimalModbus using DTB4824.

Test the source distribution generation (look in the resulting PKG-INFO file):

make dist

Also make sure that these are functional (see sections below):

Documentation generation

Test coverage report generation

Git¶

Merge to the master branch locally. Make a tag in the git repository. Push the master branch (including tags) to Github.

If you push to another branch, a pull request will be generated.

See below for details.

GitHub¶

Releases are automatically generated on GitHub from tags in the repo.

Upload to PyPI¶

Build the source distribution and wheel, and upload to PYPI:

make dist
make upload

Test the documentation¶

Test links on the PyPI page. If adjustments are required on the PyPI page, log in and manually adjust the text. This might be for example parsing problems with the ReST text (allows no Sphinx-specific constructs).

Force documentation rebuild on readthedocs¶

Log in to https://readthedocs.org (using Github credentials) and force rebuild on the master branch.

Enable the “master” and “stable” documentation versions. In the advanced settings select Python3.

Test the installers¶

Make sure that the installer works, and the dependencies are handled correctly. Try at least Linux and Windows.

On windows you might need to use:

py -m pip install minimalmodbus

Test on hardware¶

Test the package on hardware slave via Linux, Windows and Mac OS. Download the file test_deltaDTB4824.py.

To run the hardware test on Windows:

C:\Python27>python.exe C:\Users\jonas\Documents\Pythonprogram\testmodbus\test_deltaDTB4824.py -d COM7 -b 2400 -a

For python3 you might need to use the py command.

Begin a new development version¶

Check in a new version on GitHub master branch. If the previous release was X.Y.Z, then use X.Y.(Z+1)a1.

Useful development tools¶

Each of these have some additional information below on this page.

Git: Version control software. See https://git-scm.com/
Sphinx: For generating HTML documentation. See https://www.sphinx-doc.org/
Coverage.py: Unittest coverage tool. See https://coverage.readthedocs.io/
PyChecker: This is a tool for finding bugs in python source code. See http://pychecker.sourceforge.net/
pycodestyle: Code style checker. See https://github.com/PyCQA/pycodestyle#readme

Git usage¶

Clone the repository from GitHub (it will create a directory):

git clone https://github.com/pyhys/minimalmodbus.git

Show details:

git remote -v
git status
git branch

Stage changes:

git add testb.txt

Commit locally:

git commit -m "test1"

Commit remotely (will ask for GitHub username and password):

git push origin

Git branches¶

Create a new branch:

git branch develop

List branches:

git branch

Change branch:

git checkout develop

Commit other branch remotely:

git push origin develop

Make a tag in Git¶

See the section on Git usage.

The release is done in the ‘master’ branch, not the ‘develop’ branch. List tags:

git tag

Make a tag in Git:

git tag -a 0.7 -m 'Release 0.7'

Show info about a tag:

git show 0.7

Commit tags to remote server:

git push origin --tags

Sphinx usage¶

This documentation is generated with the Sphinx tool: https://www.sphinx-doc.org/

It is used to automatically generate HTML documentation from docstrings in the source code. See for example Internal documentation for MinimalModbus. To see the source code of the Python file, click [source] on the right part of that page. To see the source of the Sphinx page definition file, click ‘View page Source’ (or possibly ‘Edit on Github’) in the upper right corner.

To install, use:

sudo pip3 install sphinx sphinx_rtd_theme

Check installed version by typing:

sphinx-build --version

Spinx formatting conventions¶

What	Usage	Result
Inline web link	`Link text <http://example.com/>`_	Link text
Internal link	:ref:`testminimalmodbus`	Internal documentation for unit testing of MinimalModbus
Inline code	``code text``	`code text`
String	‘A’	‘A’
String w escape ch.	(string within inline code)	`'ABC\x00'`
(less good)	(string within inline code, double backslash)	`'ABC\\x00'` For use in Python docstrings.
(less good)	(string with double backslash)	‘ABC\x00’ Avoid
Environment var	:envvar:`PYTHONPATH`	`PYTHONPATH`
OS-level command	:command:`make`	make
File	:file:`minimalmodbus.py`	`minimalmodbus.py`
Path	:file:`path/to/myfile.txt`	`path/to/myfile.txt`
Type	`bytes`	bytes
Module	:mod:`minimalmodbus`	`minimalmodbus`
Data	:data:`.BAUDRATE`	`BAUDRATE`
Data (full)	:data:`minimalmodbus.BAUDRATE`	`minimalmodbus.BAUDRATE`
Constant	:const:`False`	`False`
Function	:func:`._checkInt`	`_checkInt()`
Function (full)	:func:`minimalmodbus._checkInt`	`minimalmodbus._checkInt()`
Argument	`payload`	payload
Class	:class:`.Instrument`	`Instrument`
Class (full)	:class:`minimalmodbus.Instrument`	`minimalmodbus.Instrument`
Method	:meth:`.read_bit`	`read_bit()`
Method (full)	:meth:`minimalmodbus.Instrument.read_bit`	`minimalmodbus.Instrument.read_bit()`

Note that only the functions and methods that are listed in the index will show as links.

Headings

Top level heading underlining symbol: = (equals)
Next lower level: - (minus)
A third level if necessary (avoid this): ` (backquote)

Internal links

Add an internal marker .. _my-reference-label: before a heading.
Then make an internal link to it using :ref:`my-reference-label`.

Strings with backslash

In Python docstrings, use raw strings (a r before the tripplequote), to have the backslashes reach Sphinx.

Informative boxes

.. seealso:: Example of a **seealso** box.
.. note:: Example of a **note** box.
.. warning:: Example of a **warning** box.

Sphinx build commands¶

To build the documentation, in the top project directory run:

make docs

That should generate HTML files to the directory docs/_build/html.

TODO¶

See also GitHub issues: https://github.com/pyhys/minimalmodbus/issues

Possibly use pytest instead
Improve installation troubleshooting
Test virtual serial port on Windows using com0com
Unittests for measuring the sleep time in _communicate().
Tool for interpretation of Modbus messages