Status: Draft for comment

This is a proposal for how to solve the grbl versioning problem.

Audience:

Authors of grbl-compatible controllers and senders.

Problem Statement:

There are many variants of grbl-inspired firmware including the
“original” grbl 0.x and 1.x, adaptations of that for specific
machines, ports and extended versions like grbl-Mega, grbl-LPC,
grblHAL, Grbl_Esp32, and almost-compatible firmware like Smoothieware.

On the other side, there are innumerable senders that can talk to
some subset of the variants.

Most grbl variants tend to be quite compatible with the core protocol,
so that a given sender is very likely to be able to drive most
variants, if only it were possible to determine that the controller
implements the grbl protocol, and to handle any variant-specific
dialect differences from classic grbl.

This proposal addresses two problems:

1. Grbl Detection: How does a sender detect that it is connected to
   a device that implements the general grbl protocol, as opposed
   some random serial device or a controller firmware that uses a
   different protocol, for example g2 or Marlin.

2. Variant Handling: How does the sender determine which variant is present
   and thus adapt to its dialect differences. There are two sub-problems
   here – the first is identifying the variant and the second is determining
   the dialect implications of a given variant.

Assumptions and Goals:

• The goal is to enhance interoperability between senders and controllers,
  particularly going forward as new capabilities are being developed.
• Compatibility with existing senders should be preserved where feasible.
• There may be some senders whose interpretation of the grbl protocol
  is so rigid that even the slightest change will break them, that
  are no longer supported and cannot be modified. Supporting such
  senders is on a “best effort” basis, but should not compromize the
  goal of solving the problem for mainstream senders. Some such senders
  are already broken, in the sense that they fail to recognize some
  grbl variants.

Existing grbl controllers typically issue a welcome message that
looks something like

Grbl 1.1d ['$' for help]

Ideally, a sender could detect such a message and use it to detect
that grbl is present. Unfortunately, there are at least three
problems:

• Different grbl controllers vary the message in ways that can confuse
  a simple parser that is trying to detect such a message. Here are some
  welcome message variants that have been seen in the wild; there are
  probably others.

    Grbl 0.9j ['$' for help]
    Grbl 1.1d ['$' for help]
    Grbl 1.1
    Grbl 1.1h: LongMill build ['$' for help]
    Grbl 1.1h ['$' for help] LongMill build Feb 25, 2020
    gCarvin 2.0.0 ['$' for help]

• The welcome message typically attempts to solve two different problems,
  namely detection and variant identification. Given the lack of a clear
  specification for either, authors of grbl variants have adopted different
  message variations, making it difficult for senders to know what to expect.

• The welcome message is typically issued only when the controller
  first boots or when it is reset with Ctrl-X. In classic grbl, this
  was not a problem because Arduino hardware typically resets when a
  serial connection is established. In newer scenarios, on
  microprocessors that do not reset on serial connection or where
  connections are made via other channels, the reset-on-connection
  assumption is questionable. (Even with reset-on-connection, there
  is still a possible timing problem related to the time it takes for
  the controller to start compared with the time it takes for the
  serial connection to become stable and to discard possible garbage
  characters on the line.) This leads to the question of how a sender
  can ensure that a detectable message is issued. One solution is to
  send a Ctrl-X reset if no message is seen within a certain time. That
  isn’t ideal, because reset will kill any job that is currently in progress.
  That wasn’t a problem with classic grbl where the sender actively transmits
  the gcode file, but could be a problem with modern grbls that have local
  SD cards and can run jobs when the UI is not connected.

The goal of this section is allow a sender to determine, quickly and
unambigously, that it is connected to a device that implements the
grbl protocol.

A grbl controller shall issue a welcome message, formatted as below,
in the situations specified below.

The welcome message shall be of the form:

Grbl N.Ml trailer

“N” and “M” are single decimal digits. “l” is an optional lower-case
letter between “a” and “z”. “trailer” is an optional arbitrary string
of characters. If trailer is present, the space before it is mandatory.
If trailer is not present, the space after “l” is optional.

A sender shall detect any string of that form as an indication that a
grbl-compatible controller is present. A sender may detect other
strings as an accomodation to legacy grbl controllers that existed
prior to this specification.

A sender may distinguish between N=0 and N=1 to detect the grbl protocol
changes that occurred between classic grbl 0.9 and 1.1 – but there are
better ways to deal with some of those protocol changes, as outlined below.

The trailer may contain identification information to specify which
grbl variant is present. A sender may use that information to support
legacy grbl controllers. New controllers shall not depend on the
trailer for precise identification, instead using the identification
methodology below.

The welcome message shall be issued in these situations:

1. When the grbl controller starts, it shall issue a welcome message
   on all active interfaces that are intended for use with grbl serial
   protocol. An active interface is one that is currently ready for
   communication, for example a serial port or a network connection that
   is automatically established during welcome.

2. When a new connection is established on an interface that is
   intended for use with the grbl serial protocol, the grbl controller
   shall issue a welcome message.

3. When the grbl controller is reset via Ctrl-x, it shall issue a
   welcome message

There are situations where the grbl controller cannot detect connection
establishment – for example a USB-to-serial adapter that does not
automatically reset the microprocessor on a new connection. To accomodate
such situations, senders should implement a fallback method for resetting
the grbl controller, either by issuing a Ctrl-X reset command upon explicit
user request, or by requiring the user to manually reset the grbl controller.
A sender should not issue Ctrl-X automatically, unless it first resorts to
other means of discovering the possibility that a grbl controller is present
and running. An example of such a method would be to issue a “?” status
report request and look for a grbl-compatible response.

The goal of this section is allow a sender to identify which grbl
variant it is connected to. The sender must first determine that
it is connected to a grbl-compatible device according to the preceding
section.

The existing grbl protocol contains a [MSG: …] form. Such messages
can be issued at any time. They are not necessarily synchronized to
any particular query. The grbl wiki lists the specific messages that
are supported by grbl 1.1. Some other existing controllers add additional
messages to provide additional information. The expectation is that
senders will report those messages to the user in some fashion. The
further expectation is that senders should be prepared to receive such
a message at any time.

We propose to add additional formal structure to the [MSG: …] form,
to be used to convey identification information in a standardized
manner.

A message of the form [MSG:tag: value] is a “tagged message”. “tag” is
a string of non-blank characters from the set A through Z, 0 through 9,
and underscore (‘_’). “tag” must be followed by “: ” (colon then space).
“value” is an arbitrary string of printable characters not containing
the character ‘]’. The permissible values of “value” depend on tag.

If “tag” begins with underscore, it is a “standard tag” whose meaning
is agreed upon by a group of parties including controller and sender
author/maintainers. Such standard tags, along with their associated
sets of value, can be used to convey information that is commonly
useful across the spectrum of different controller and senders.

_FW: The _FW tag identifies the particular grbl variant by its
common name. Example values are grblHAL and Grbl_Esp32. Each author
of a new grbl variant may choose their own name, so long as it does
not conflict with an existing name. Example: [MSG:_FW: Grbl_Esp32]

_URL: The _URL tag provides a reference to further information about
the grbl variant. Its value is a standard URL, for example [MSG:_URL: https://github.com/gnea/grbl]

_VER: The _VER tag identifies the version of the firmware. Versions
have meaning only within the context of a variant; there is no
expectation that versions will be coordinated across different
variants. Authors are responsible for defining their own versioning
scheme – but, in keeping with current industry practice, we suggest
some form of “semantic versioning” like “v2.0.1” with a possible build
suffix or git hash like “v2.0.1-23f98e7”

_DATE: The _DATE tag gives the build date or release date of the
firmware, in W3C format as given by https://www.w3.org/TR/1998/NOTE-datetime-19980827 . Example: [MSG:_DATE: 2020-13-03T13:15:30Z

(Reviewers: Please suggest additional tags that may be useful/needed)

Tags that do not begin with underscore are private tags whose meaning
depends on the grbl variant, and are not coordinated across variants.
Thus two different variants might use the same private tag name with
different values.

Identification messages including any _FW, _URL, _VER, and _DATE
tagged messages that the grbl firmware supports shall be issued in
these situations:

1. After a welcome message is issued.

2. After the response to $I – but after the “ok” that acknowledges
   the execution of $I. (Note: it is unclear whether it is in fact
   necessary to delay the identification message until after the “ok”,
   in light of the grbl stipulation that [MSG…] can appear at any
   time.)

This section recommends practices that can help senders to be more
robust in the presence of different grbl variants and future evolution
of the grbl protocol.

Since different variants and their versions sometimes have protocol
implications, it is tempting to look at the version as an indicator
of how to handle protocol details. The difficulty is that it is
very difficult, bordering on impossible, to manage version numbers
across a range of variants by different authors. Where possible,
it is better to detect the protocol variation by direct inspection of
the message. A good example is the handing of “error:” messages.
In grbl 0.9, the format was “error:text describing the error”, while
grbl 1.1 changes to “error:X” with X being a number. It is relatively
easy for a sender to accept either form, without regard to version number.

Another example involves the “option codes” in the classic grbl [OPT: ]
message. Those options denote which features have been enabled at
compile time. The difficulty is that, with the rate at which new
features are being added to grbl variants, it is difficult/impossible
to coordinate the list of options and their meanings, especially across
different variants with different authors. Thus, where possible, it
is better to detect such features by “asking specific questions”, for
example issuing a GCode command that depends on the feature and checking
for an error response.

With the advent of grbl variants running on microprocessors that break
through the memory limits of the early AVRs that originally hosted grbl,
many new configurations can be supported. The classic set of numbered
configuration parameters is inadequate, difficult to extend, difficult
to coordinate extensions across variants, and hard for humans to remember
without always resorting to table lookup.

Thus it is probably prudent for general-purpose senders not to make too
many assumptions about the exact set of numbered configuration parameters.
For specific known variants, a sender might well know which parameters
are supported by a given version, but maintainting that in the long term
is likely to be a never-ending battle.

A better approach is for controllers to provide a mechanism whereby
the sender can inquire about the set of configuration options, with
the results returned in a self-describing human-understandable format.
(This might become the topic of a future proposal.)

Author:
The author of this proposal, Mitch Bradley, is a developer/contributor to the CNCjs sender, the Grbl_Esp32 controller firmware, and the g2core controller firmware.