System iNetwork

Refactor DtaQExample3Thread into a generalized, generic form

The new class is ModelOutputDataQueue. This class will handle the data queue output of a Model (MVC), any Model of which follows the design presented. In order to do this anything which ties the model's output data queue handling to the specific Demonstration Model has to be moved out of this class. This class will work with the demonstration Model running on the System i and future Models, given some help outside the new generalized class.

In fact this is a very generalized keyed data queue reading class which can be run in a Thread. Considered in isolation, it is independent of the whole notion of MVC. It could be used anywhere that a keyed data queue needs to be read from a Thread.

Many of the design aspects for DtaQExample3Thread are retained. The changes are:

1. The new class doesn't assume anything about the data being read from the data queue. Reading the keyed data queue returns a KeyedDataQueueEntry, a generic object which assumes nothing about the data or its format. Taking the data from the KeyedDataQueueEntry and putting it into a Record is done outside the new class (by the Listener). A generalized, reusable class won't know what the data which the model will send looks like, the data structures may not have even been designed.

2. Give the class a way to stop the run() method short of crashing the data queue. This is done by the method stopReadingDq(). This method sets a flag which breaks the run loop then writes a dummy entry to the data queue which the run() loop is reading. The run() loop reads the dummy entry, sees the flag is set and breaks out without firing the PropertyChangeEvent.

A side note: A refinement would be for the PropertyChangeEvent to fire a "dqEnded" event before breaking the loop. That way the PropertyChangeListener could act on the fact that the program is no longer reading the Model's output data queue.

3. Every 10 minutes the read() times out then re-read the data queue. This gives the program a chance to throw an error if the data queue isn't available.

4. Create Exception stubs for everything the System i could throw. Later fill in as needed.

5. Consolidate exception reporting into one method. Right now this method simply does a printStackTrace(). In future implementation this method will be expanded to log errors and to take some action like cause the program to terminate. Remember the run() method of a Thread can't throw an exception. The reporting method will send a dqError PropertyChangeEvent to the PropertyChangeListener so it can notify the rest of the program something happened.

6. The class implements Runnable instead of extending Thread. The user of this class will create its own Thread, instantiating it with this class.

Refactor DtaQExample3a to put the data queue writing logic into a separate class

The new class is ModelInputDataQueue. This class separates the function of writing to the Model's Input Data Queue from the main application DtaQExample3d. The main application performs the functions from a specific implementation of a Model, such as the Demonstration Model running on the System i. This class only writes the data queue as DtaQExample3a did, but without any model specific details. The changes to generalize the class were done for the same reason as the Output class:

1. The new class doesn't assume anything about the Record which will be written. Instead the Record will be passed to the write() method.

2. Create Exception stubs for everything the System i could throw. Later fill in as needed. . The write() method can throw exceptions so the caller of the write() method will have to be prepared to handle (or ignore) them.

3. Consolidate exception reporting into one method. Right now this method simply does a printStackTrace(). In future implementation this method will be expanded to log errors and to take some action like cause the program to terminate.

DtaQExample3aListener becomes DtaQExample3dListener

DtaQExample3dListener required only one change. DtaQExample3aListener received a Record from the Event fired by the Thread. Record is specific to a RecordFormat which is specific to a System i file. DtaQExample3dListener receives a KeyedDataQueueEntry which is generic. The class builds a model output Record from the KeyedDataQueueEntry sent in the Event.

Only a few lines of code are changed to accomplish this. The Listener becomes very specific for the data it listens for while the Thread that is being listened to knows nothing about the data. It just reads a KeyedDataQueueEntry and passes it on to the Listener.

The PropertyChangeListener which will be attached to ModelOutputDataQueue can't be standardized into a generic class because what is does is specific to
- the Model being implemented,
- the data sent to the listener in the PropertyChangeEvent object, and
- possibly the name of the event fired.

DtaQExample3a becomes DtaQExample3d

DtaQExample3d is the main application. The ModelInputDataQueue class now has no dependencies on the specific Model that it is used with. This means that their caller (DtaQExample3d) must creates the model input Record required for the implemented Model and pass it to the write() method.

The code to write the model input data queue is removed from the main application (DtaQExample3a) and replaced by calling the write(Record) method of ModelInputDataQueue.

A Thread is created to run ModelOutputDataQueue.run() in place of DtaQExample3Thread.

To shut down the program call ModelOutputDataQueue.stopReadingDq() to end the Thread. After calling this method wait for the thread to end with Thread.join(milliseconds). After the Thread ends or the join times out close the connections to the System i and end the program with System.exit(0).

Replace Me.getKEY() with the class IpDtaQKey

IpDtaQKey generates a key based on IP address. The key represents a View-Controller which communicates with the Model. Often the VC will be running on multiple desktops. IpDtaQKey creates a key for the model data queues from the IP address of the desktop client. This will be unique per desktop.

Another side note: One caveat though! This breaks down if two or more clients were to run separately on the same desktop using the same set of data queues. In this case the key would have to be "uniquified" for each client (on the same desktop using the same data queues).

Java is not well suited for this. Each JVM lives in its own world and is unaware of any others which may be running on the same system. The JVM can't make use of the operating system resources for coordination because the JVM is intentionally blind to operating system specifics.

To solve this I'd use this approach.

1. Decide on a folder any JVM on the desktop can find. Make sure it is not dependent on the user logged on, that is not dependent on the users home directory. For example C:\TEMP.

2. Create a Collection object of "unique" values of 0 to 255 which can be represented by x00 to xFF. The unique value will be the look up if you choose a Map type Collection. If you choose an Array type collection than the unique value will be the index. I'd also put a timestamp on it so old value could be recycled if they hadn't been removed.

3. Each time the program which needs a key starts up it
- calls the IpDtaQKey for the base key,
- deserializes the Collection,
- finds a unique value and convert it to hex (2 characters),
- updates the Collection with the unique value used,
- cleans up any obsolete used unique values (from the timestamp),
- then serializes the Collection.
The data queue key for this instance is IpDtaQKey.getDqKey() + uniqueValue

4. When the program ends deserailze the Collection, release the unique value so it can be re-used and serialize the Collection. The clean up in the previous step is a back up because programs crash and you can't depend upon this step being executed each time a new unique value is put in the Collection.

Like I said, Java doesn't make it easy. Please let me know if you have better ideas. This is an on-going problem for me. What I have described is the best solution I have come up with to date.

If you run these examples in QSHELL on the System i then you need to do something else for the key. I have started a dummy job, gotten its number, ended the dummy job and used the dummy job's job number for the key.

Try to avoid a design which requires the key value to be created by the user in a property or environment variable. This works great during development but when you start distributing the application you will find that maintaining a unique key per installation is a real pain. Been there, done that. Bad!

Play with running both the CL VC and DtaQExample3d!

Actually you can do this with any of the DtaQExample programs. The CL demo View-Controller can write an entry to the demo Model output data queue without reading it back. If you do this while DataQExample3# is running, with the same key the instance of DtaQExample3# is using, the instruction and data will pop up on the console.

All the entries on the demo model data queue are being placed on it by the demo Model running in batch on the iSeries. The difference is where the input to the Model came from, the System i CL program on the desktop Java application. DtaQExample3# shows both and doesn't treat them any differently.

The thread to read the KeyedDataQueue starts to show its power when messages pop up on the console from the model while the console is reading stdin for your input.

The code is in the CVS

The new classes have been added to the dataqueue package except for IpDtaQKey which was added to the common package.

Taking some time off

The next blog entry set will be about writing an Eclipse RCP demonstration View-Controller that has been implemented as
- a CL program
- as a Java console program (6 different ways now)

I am experimenting with moving my Eclipse 3.2 work into WDSc 7. If these experiments are successful then the next exercise will be done in WDSc 7 and include notes on how to do it in Eclipse 3.2.

Right now my personal preference is native Eclipse. However WDSc 7 has advanced to the point that managing two separate development environments may outweigh the extra capabilities of native Eclipse. I will see and get back to you on this.

I am taking some time of posting this blog to take care of this and other things.

In the mean time please write if you have questions or comments. Please write to tell me what you'd like to see in an Eclipse RCP application version of this Java console program. Tell me what parts you would like explored in detail.

Thanks all. Enjoy the spring -- that is part of my "other things" :)