Operating
Serial Communications
under Visual dBASE 7.01
by Nicolas Martin
including a short history of the origins of Serial Communications.

Introduction

Many databases applications, like those developed under Visual dBASE, are designed for a multi-user environment, where several clients share a common database. This type of distributed architecture depends upon the services provided by a network to interconnect several machines so that they can share and exchange information

To cope with particular requirements, it is sometimes necessary to connect a specific equipment to a local client machine such as a modem, and to exchange data with only this equipment. Different solutions are available for such connections, the best known types are parallel and serial connections. In this article, we are going to consider serial communications, available on all PCs, and compliant with the RS232-C standard.

This type of interface has been around since the beginning of the computer era, and, in many technical domains, remains a good and efficient way to perform point to point connections between two computers or equipment. Even now, in the Internet and worldwide network age, many users still connect to the world wide web using a modem that is linked to their computer with a “serial” cable using an RS232 interface.

Accordingly, computer applications — including Visual dBASE’s — are sometimes required to operate this RS 232 serial link to exchange data with equipment or another computer hooked to the machine on which it is running.

This article, while introducing a new release of a Visual dBASE component named SerialComm 1.21 (attached to this document) presents the fundamentals of serial communications, their specific operational requirements on PC computers, their use from the Win32 operating system, and the integration of the SerialComm custom class component in a Visual dBASE application.

The article is divided into the following chapters:

 
   

As a general introductory topic, a short retrospective of the origins of serial communications presents some of the more interesting facts that drove the transmission techniques from their discovery to modern technologies.

Hardware basics

This chapter will give a simple overview of the hardware basics needed to understand the way a serial communication port works, from a software point of view. A full description of the hardware methods and details to connect and operate the serial port is given in a separate document, hardware description, but it is not necessary to read it to understand the present article. It is included for the interested reader, so that all the topics from hardware to software are covered.

DTE and DCE Devices

Two terms are commonly used in serial communications, DTE and DCE. DTE stands for Data Terminal Equipment, and DCE stands for Data Communications Equipment. Your computer is a DTE device, while most other devices are usually DCE devices. A DTE is connected to a DCE with a straight cable, while two like devices must be connected with a cross wire one. See hardware description for more information about serial cables.

You can simply replace the term “DTE device” with “your PC” and the term “DCE device” with “remote device” in the following discussion. The following table gives the signals involved in the serial communication port:

Abbreviation Full Name Function
TD
Transmit Data
DTE Serial Data Output (TD)
Data is sent from a DTE device to a DCE device on this wire. 
RD
Receive Data
DTE Serial Data Input (RD)
A DTE device receives data on the RD (receive data) wire.
RTS
Request To Send
The DTE device puts this line in a mark condition to tell the remote device that it is ready and able to receive data. If the DTE device is not able to receive data (typically because its receive buffer is almost full), it will use this line as a signal to the DCE to stop sending data. When the DTE device is ready to receive more data (i.e., after data has been removed from its receive buffer), it will place this line back in its original state.
CTS
Clear To Send
The complement of the RTS wire is CTS. The DCE device uses this line to tell the DTE device that it is ready to receive the data. Likewise, if the DCE device is unable to receive data, it will change this line’s state. 
DSR 
Data Set Ready
DSR (Data Set Ready) is the companion to DTR in the same way that CTS is to RTS.
SG
Signal Ground
 
CD
Carrier Detect
A modem uses Carrier Detect to signal that it has made a connection with another modem, or has detected a carrier tone.
DTR
Data Terminal Ready
Its intended function is very similar to the RTS line. Some serial devices use DTR and DSR as signals to simply confirm that a device is connected and is turned on. The DTR and DSR lines were originally designed to provide an alternate method of hardware handshaking.
RI
Ring Indicator
A modem toggles the state of this line when an incoming call rings your phone.

Note: The Carrier Detect (CD) and the Ring Indicator (RI) lines are available only in connections to a modem. Because most modems transmit status information to a PC when either a carrier signal is detected (i.e., when a connection is made to another modem) or when the line is ringing, these two lines are rarely used.

Full duplex versus half duplex communications

Achieving a half duplex or a full duplex communication process can depend upon the transmission media, or the way the application handles the incoming stream of data.

Performing full duplex communication is generally better because it is more efficient in terms of effective data flow, and does not require any particular protocol to arbitrate the channel. However, it requires the capability of simultaneously sending data and receiving incoming data be present at both ends.

Flow control, handshake

Flow control is needed when the application that sends data transmits the data at a higher rate than the application that receives this data can process it. For instance, if our DTE to DCE speed is several times faster than our DCE to DCE speed, sooner or later data is going to get lost in the DCE as buffers overflow. Flow control has two basic varieties: hardware or software.

Even though these two types of flow control can achieve the same results, it is generally better to use hardware flow control because it is more efficient. On the other hand, using both flow types of control is generally useless.

Asynchronous communications

Let’s consider the synchronization problem: the receiver is going to track the state of its receive wire line, and will see a succession of state changes, “0”s and “1”s, from which it will have to recognize the characters.

There are two possible ways to decode the incoming bitstream:

An asynchronous line that is idle is identified with a value of 1, also called a mark state. By using this value to indicate that no data is currently being sent, the devices are able to distinguish between an idle state and a disconnected line. When a character is about to be transmitted, a start bit is sent. A start bit has a value of 0, also called a space state. Thus, when the line switches from a value of 1 to a value of 0, the receiver is alerted that a data character is about to come down the line.
Transmission parameters and errors

The Start bit

After receiving the start bit, the receiver can synchronize an internal clock, which depends on the line speed. The clock keeps tabs on the timing and samples the line at evenly spaced instants to determine the value of the transmitted bits.

The Data bits

There may either be 5, 6, 7, or 8 data bits, depending on the physical line configuration parameters. Both the receiver and the transmitter must agree on the number of data bits, as well as the baud rate. Almost all devices transmit data using either 7 or 8 data bits.

Notice that when only 7 data bits are employed, you cannot send ASCII values greater than 127. Likewise, using 5 bits limits the highest possible value to 31.

Baud Versus Bits Per Second

The baud unit has its origins in Jean Maurice Emile Baudot, who was an officer in the French Telegraph Service. He is credited with devising the first uniform-length 5-bit code for characters of the alphabet in the late 19th century. What baud really refers to is modulation rate or the number of times per second that a line changes state. This is not always the same as bits per second (BPS). If you connect two serial devices together using direct cables then baud and BPS are in fact the same. Thus, if you are running at 19200 BPS, the line is also changing states 19200 times per second. But when considering modems, this isn’t the case.

Because modems transfer signals over a telephone line, the baud rate is actually limited to a maximum of 2400 baud. This is a physical restriction of the lines provided by the phone company. The increased data throughput achieved with 9600 or higher baud modems is accomplished by using sophisticated phase modulation and data compression techniques.

The speed defines explicitly the size of each individual bit. In theory, any speed is possible, but only the following ones are available on a PC:

Available speeds, in bits per second (or baud)
110
4800
56000
300
9600
57600
600
14400
115200
1200
19200
128000 (*)
2400
38400
256000 (*)

(*) The speeds of 128000 and 256000 baud are not available on all hardware, and depend upon the type of circuit used.

The Stop bit(s)

After the data has been transmitted, a stop bit is sent. A stop bit has a value of 1, a mark state, and it can be detected correctly even if the previous data bit also had a value of 1. It is recognized because of the stop bit’s duration. Stop bits can be 1, 1.5, or 2 bit periods in length.

There can be different combinations of number of data bits and number of stop bits, presented in the following table:

Number of Data bits
Number of stop bits
8
1 or 2
7
1 or 2
6
1 or 2
5
1 or 1.5

The data sent using this method is said to be “framed”, i.e., the data is framed between a Start and Stop Bit(s). Should the Stop Bit(s) be received as a Logic 0, then a framing error will occur. This is a common occurrence, when each side is communicating at a different speed.

The parity bit

Besides the synchronization provided by the use of start and stop bits, an additional bit called a parity bit may optionally be transmitted along with the data, just between the data bits and the stop bit(s). A parity bit affords a small amount of error checking, to help detect data corruption that might occur during transmission. You can choose either even parity, odd parity, mark parity, space parity or none at all. When even or odd parity is being used, the number of marks (logical 1 bits) in each data byte are counted, and a single bit is transmitted following the data bits to indicate whether the number of 1 bits just sent is even or odd.

For example, when even parity is chosen, the parity bit is transmitted with a value of 0 if the number of preceding marks is an even number. For the binary value of 0110 0011 the parity bit would be 0. If even parity were in effect and the binary number 1101 0110 were sent, then the parity bit would be 1. Odd parity is just the opposite, and the parity bit is 0 when the number of mark bits in the preceding word is an odd number. Parity error checking is very rudimentary. While it will tell you if there is a single bit error in the character, it doesn’t show which bit was received in error. Also, if an even number of bits are in error, then the parity bit would not reflect any error at all.

Mark parity means that the parity bit is always set to the mark signal condition and likewise space parity always sends the parity bit in the space signal condition. Since these two parity options serve no useful purpose whatsoever, they are almost never used. SerialComm proposes however the “Mark” parity as a possible choice.

In summary, the parity bit follows the following rules:

Parity type
Description
None
No parity bit added
Odd
if number of 1 in the received character is odd, then parity bit is 0.
Even
If number of 1 in the received character is even, then parity bit is 0
Mark
Logic 1 (Idle state)

Each time a character is received by the receiving hardware, if the parity check is enabled, the hardware will recompute the corresponding parity and compare it with the parity bit received. If they are identical, the hardware declares it correct. If they are different, the hardware declares a parity error.

The Break Signal

Last, we can introduce the notion of  “Break” Signal. This is when the data line is held in a Logic 0 state for a time long enough to send an entire character. Therefore if you don’t put the line back into an idle state, then the receiving hardware will interpret this as a “break” signal.

A sample waveform

The diagram below shows the expected waveform of a character when using 8 Data bits, No Parity and 1 Stop Bit. The RS-232 line, when idle, is in the Mark State (Logic 1). A transmission starts with a start bit which is Logic 0. Then each bit is sent down the line, one at a time. The LSB (Least Significant Bit) is sent first. A Stop Bit (Logic 1) is then appended to the signal to make up the transmission.

The diagram shows the next bit after the Stop Bit to be Logic 0. This must mean another word is following, and this is its Start Bit. If there is no more data coming, then the receive line will stay in its idle state (Logic 1).

The above waveform applies to the Transmit and Receive lines on the RS-232 port that carry serial data. The other lines on the RS-232 port which, in essence are parallel lines (RTS, CTS, DSR, DTR, CD and RI) are also at RS-232 Logic Levels.

Architecture of a Communication application under Windows

Generally speaking, a communication application will be in charge of:

This can be viewed also as sending and receiving, because managing input lines or incoming Xon/Xoff characters can be considered as a receiving process, while managing output lines or sending Xon/Xoff characters can be viewed as a sending operation.

Even if the two processes — sending and receiving — look rather symmetric and similar, from an algorithmic point of view, there are major differences bewteen them:

 
Polling
Interrupt
Description
Consists of polling the communication port and status lines to check whether new data is available or if a control signal has changed.  Consists of calling the function that reads the communication device - and thus the new incoming character or signal state change - only when the communication device hardware signals it by generating a hardware interrupt. The function called when an interrupt occurs is named an “Interrupt Service Routine”, or ISR.
Advantages
Easy to implement. Best method to deal with communication processes,  because ISR is called only when needed.
Drawbacks
Very much CPU time consumed.
High data flow rates cannot be achieved directly by this method.
More difficult to implement, debug, and maintain.
Hardware closely dependent.

During the good old times of DOS applications, use of an interrupt-driven application was available only to assembly or C programmers, because they could directly access all the hardware resources of the machine. The drawbacks of such applications were that there was a high level of software complexity, they were difficult to debug, and they provided low maintenance abilities.

In a few words, an interrupt is a way to inform and signal a microprocessor which is executing a software program in memory, that external hardware events occur. An interrupt line corresponds to a hardware input, supervised by the microprocessor. When it becomes active, the microprocessor immediately stops its current execution to process another program (called the interrupt handler or the interrupt service routine) before returning to the code it was previously executing before the interruption arrived.

The arrival of Windows has swept away all these low level techniques, as this operating system controls the management of all the hardware resources of the machine. Applications must no longer directly address the hardware resources. An application addresses Windows, which, in turn, operates any hardware resource, especially the communication ports.

At first glance, this may look like a regression, because an interrupt handler cannot be anymore settled as before. But in fact, Windows kernel sets up already, to communicate with the serial ports, an interrupt process and fills up, when characters are received, an input buffer that the application can read when necessary. Let’s look in greater detail how the communication application is going to be designed now under Windows:

 
Polling-like method
Interrupt-like method
Description
Ask Windows to get the next characters from the receive queue, extracting them from the FIFO buffer, and the port status. This is different from direct polling on the COM port, as discussed before, because here several characters can be read at a time. If characters are read 10 by 10, the read function is called at a tenth the maximum character frequency, and executes 1/10 times the same application code. Windows is configured so that it signals the application the events that occur on the communication port. In Windows 16 bits, this was achieved using WM message. In Windows 32 bits, it uses thread synchronization and events. Both act like an interrupt process, with the difference being that the ISR will execute here only when the operating system activates.
Advantages
Easy to implement.
Application code can be executed at a much lower frequency than the character frequency and can thus afford to be slower.
Most efficient method in terms of latency delay.
Port status changes can be signaled.
Drawbacks
Requires a receiving queue long enough to store temporarily received characters between each read.
Latency delay between the reception and the processing of characters; on the average, this delay is 1/2 the period at which the queue is read.
Port status needs to be checked regularly.
Difficult to implement, because it requires multi-threading or message queue process by callback functions.
Application’s receiving code executed at high frequency, requires optimized and fast application code.
Stability within the O.S.

A simplified view of the communications scheme under Windows can be illustrated with  the following diagram.

The Windows kernel sets up its Application Programming Interface (API) with which the application (in our case, the Visual dBASE program) dialogs. The Windows kernel also sets up the Receive (Rx) queue and the Transmit (Tx) queue, which can be accessed both by the Windows API functions, and by Windows Interrupt handlers. Those handlers work asynchronously with the application. They can dialog synchronously, through the hardware drivers, with the hardware, which in turn drives the physical line. The hardware can also signal events (that can be: new character received, control line state change, ...) to the interrupt handlers and to a control API, which in turn translates them into events to the application.

Serial Communication under Visual dBASE 7 using the SerialComm 1.21 package

Attached to this article is the SerialComm 1.21 software package that you can easily retrieve with the link at the bottom of this page. Here is the list of the Visual dBASE files it contains, along with a short description of each of them:

Each file is self documented and contains detailed instructions of its use. This documentation is in each file’s header comment.

It is recommended, but not required, that you include the SerialComm component in your component palette, at least during the time you’re getting used to it and to the demo forms.

The four demo forms included in the package and listed above are of various levels of complexity. I recommend that you run and examine these demo programs in the following order:

  1. OneShotDemo.wfm is the simplest one.
  2. ContReadDemo.wfm shows simply the continuous read mechanism.
  3. FullDuplexDemo.wfm, with which you can start exchanging data with a host.
  4. TTYDemo.wfm, the most sophisticated, has many features that call most of the SerialComm component’s built-in functions. This form will be taken as a working example to illustrate the use of the SerialComm component later in this document.
But before getting into the demo form’s content, let’s look first in detail at how to use the SerialComm component.

Using the SerialComm 1.21 component

Now that the main substance concerning serial communications has been explored, let’s go back to our business applications and their need to communicate on serial communications.

Obviously, the choice between the polling-like and the interrupt-like methods will be determined by your needs:

If you are in the first situation, then you should forget about Visual dBASE, go to something like C or C++ and use interrupt-like methods.

On the other hand, if your application deals mostly with data exchange through the RS-232C media, with rather weak real- time requirements, then the SerialComm component, using a polling-like method should be your choice.

This SerialComm component is contained in a custom class named SerialComm.cc. It contains the required properties and methods to operate a serial communications handler using the Windows API. It’s written totally in Visual dBASE 7 code, and provides an easy way to design your communication application under Visual dBASE.

The following description is a general guideline on how you should use this component and design your application, taking into account the general interface parameters. For details on a given parameter, please refer to the component’s documentation itself, contained in the source code’s comment header.

Let’s now go through the steps to design the communication application:

An application of the SerialComm 1.21 component: a mini TTY console

After having overviewed all the SerialComm component’s possibilities, let’s now turn to an application that uses this component. We’ll go over the main implementation techniques needed to get the best efficiency and results from it.

The application proposed here is a mini TTY console, which is a kind of terminal emulator that will be enabled to receive and send data in full duplex mode, and display this data in an editor window. This may remind you in some way the “hyperterminal” application that ships with Windows. Its name is TTYdemo.wfm, and it is delivered as part of the SerialComm 1.21 package. (See the link at the end of this article).

This form has two pages. The first one, shown on startup, presents a screen editor which is used both to display the incoming data from the serial port and to enter data to be sent to the same port. The following image presents an example of a session which was obtained while connecting to an ISP provider through a modem. The commands entered on the keyboard are:

In addition to the screen editor, the top of the window contains a set of comboBoxes to choose the communication port and physical line parameters, and a red light that turns to green if the communication port is successfully opened. The bottom of the window has some buttons for specific actions:

Clicking on the Errors and Status button will display the following screen:

On the second page, the Received Errors box contains a set of received error counters. The Receive Status box indicates if an Xoff has been sent as well as the count of unread characters in the receive queue. A Transmit Errors and Status box tells if the transmission is possible or blocked, and, in that case, the number of unsent characters. A set of buttons will also perform:

All these components are set up with standard Visual dBASE techniques, and we will not get into more detailed explanations here. On the other hand, we will take an in-depth look at how the different steps described in the Using the SerialComm 1.21 component chapter are used here in this typical communication application.
First, the port is closed by calling the form’s canClose event. Then it is reopened by calling the form’s onOpen event. When reopening the communication port, the new line parameters selected will be taken into account. Line 756 also sets up the selected handshake method.
Conclusion

RS232, as a serial communication standard available on all PCs, still remains a good alternative for exchanging point to point information between computers or equipment.

Designing a software application that handles the serial communication ports has never been an easy task, even during the DOS age, as the algorithms necessary to set up have to cope with external hardware events. Moreover, timing aspects involved in such applications often requires notions of real time software programming.

Windows has now slightly changed this landscape, as it manages and controls all the hardware resources of the machine. In fact, it has masked some principles using intermediate software layers, but the basics still remain the same.

However, its kernel has opened a way to design low level like communication applications with high level development tools such as Visual dBASE. Using its high level programming language, focus can be turned to the design and conception aspects of the project. These are of prime importance in reaching a satisfactorily and efficient result.

To finish this article, I would like to say a few words about the SerialComm project. It started 1-1/2 years ago, as I needed to design an application that could connect and download data from a server through a modem and a telephone line. The first version of this component (1.01) was issued in September 1998.

At this time, I noticed that many people on the Borland newsgroup were seeking a similar tool, so I decided to share this already-designed code.

It already included many functions. But some basic areas, like status and errors report, handshaking, were absent or rather weak. As I needed to improve this part of the component for better robustness against non nominal cases in that application,  I achieved a second version of it, SerialComm 1.2, and then 1.21 while writing this article.

Additionally, after the first release of the component, I received some feedback and questions, sometimes dealing with elementary notions on operating the serial communications. I hope this article, which can be used as a general guideline on the subject, will be able to come up to those expectations.


Acknowledgments: I would like to thank especially Flip Young for her advice and their kind contribution to the proofread of this article.
To download the SerialComm 1.21 package, including the demo program, click here
(It’s a 35 Kb file)