System iNetwork

1 MVC using queues

March 15, 2007

Adding message queues to the Model (server)

It would be nice if the model (server) programs always do their "normal" or "expected" action for instructions received, but they don't. In the pure System i environment these exceptions are communicated via messages. The model (server) can also use messages and message queues to tell the view-controller (client) about these "abnormal" or "unexpected" events.

Consider a simple case, a model which adds a record to the data base.

A monolithic 5250 program would write the record and trap for an error. If an error occurred it would display information about the and let the user correct it. An error indicator would come on if the record couldn't be added (for example a second instance of a unique key), a decimal data error, NULL data was put into a non-NULL capable field, perhaps an invalid date or time value for a DATE, TIME or TIMESTAMP field.

Suppose a model program is passed the instruction to add a record to a database but encounters any of these problems. There is no interactive user to hand the error back to as in the monolithic 5250 program. How would a service program handle this situation? Message queues! The service program would probably send a message to the caller.

This data queue communicating model (server) design proposed uses message queue to communicate back to the view-controller (client). The Java toolbox does not support program to program messages queues unless the System i program was called by Java. However the Java toolbox does support external message queues.

The model program writes a message with SNDUSRMSG or SNDPGMMSG to the message queue. The Java client reads the message, can even respond to it.

That is the quick version of my model concept for MVC. If you want more information and examples please see the IBM Redbook.

This discussion is not limited to Java PC clients. The clients (view-controller) can be developed with any method, on any platform, which supports data queues and message queues. In this blog the intended clients are Eclipse RCP applications. But the "clients" could be
- 5250 applications running on the same system a different system
- Java desktop clients developed with tools other than Eclipse RCP
- C based desktop clients (Client Access provides DLLs for data queue and message queue)
- some type of web application server which communicates with the business application model though queues
- a peer-to-peer application which needs the services of your model

If your application already separates the business logic into programs you can write a data queue layer to that business logic for the Eclipse RCP client to use.

If your application doesn't already separate the business logic from the display a data queue based model then this is a good general purpose way to get the separation.

Posted by Bill Blalock on March 15, 2007 at 2:00 PM | Comments (0)

March 8, 2007

MVC for Java using data queues

The last blog entry covered the two ways Java has to call System i programs. The Java tool kit classes to call System i programs and commands are easy to use. For implementing MVC, they are too slow. The other way, JNI/RMI, is fast enough but is not easy to use. A better way is to use data and message queues to communicate between the Model (System i) and View-Controller on the desktop. This performs better than the Java client calling System i programs for each transaction. This is much simpler and faster to develop than JNI-RMI access to System i programs. In addition the data queue enabled Model will be able to work with other clients that communicate with data and message queues.

The Model is data queue enabled with one or more programs (servers) running on the System i. The data queue handling programs
- monitor input data queues for instructions from the view-controller (client), for example "read" a record, "update" a record, "execute" a transaction.
- call existing programs (the Model) to perform the business functions.
- send the results to the client (view-controller) on output data queues, for example the "record read", "status of the update", results of the "executed transaction."

One input and one output data queue is used. The input data queue is a non-keyed FIFO. The data on this queue is
- an identifier of the view-controller or client
- an instruction to the model
- data which the model is to act

The output data queue is keyed, the key is the identifier of the view-controller or client. The data on this queue is
- the instruction echoed back as confirmation
- the data that is the result of the model's action

The view-controller (client) sends the model (server) the request on the model input data queue and then reads the model output data queue by its identifier (key), waiting for the response.

Ideally business logic is isolated in callable programs, service or OPN-like. The data queue handling programs callses these programs that support the application business logic. The program processing the data queues accesses the business logic in whatever way makes sense.

The data queue handlers could be implemented as more than one program. Each program "peeks" at the input data queue (checking incoming messages) and the program which processes the "instruction" reads and removes it from the queue. An alternative would be to make the input data queues keyed and the different programs read for the specific keys that they operate on.

The model (server) program is "stateless". It executes with just the information from the input data queue, does it works, writes the output data queue, and waits for the next data queue entry. The program has no "state" from previous executions. The view-controller or client does all the remembering. This is similar to a CGI program for a HTML server and web page.

A very thorough explanation with examples of the data queue server model is presented in the IBM Redbook "Who Knew You Could Do That with RPG IV? A Sorcerer’s Guide to System Access and More" (SG24-5402-00). See section 5.2. The source for the examples can be downloaded. The instructions to download the source are in Appendix 1.

This design is scalable. One "server" program (or set of programs) can handle many clients. Additional instances of the "server" program (or set of programs) can be started if needed.

Remember, the "clients" or "view-controllers" can be anything which can communicate with the System i with data queues. Another System i, 5250 display programs, web services running on the System i and even C++ desktop clients. Use your imagination. The data queue handling programs won't know the difference nor will the model / business logic.

This design overcomes the performance problems of calling programs or commands through the Java toolbox. The model programs are running in batch on the System i. There is no start up penalty each time the client "communicates" with them.

What kind of performance can be expected?. The stock answer is "it depends."

For this blog I developed
- a demonstration model/server for the System i
- a demonstration Java view-controller for the desktop using the console (DOS prompt window)
The code for both of these will be posted in the future.

The System i model (server) is running on a Model 170. The PC Java view-controller connects to the System i through the Internet (cable modem). The model is like the data queue server described in the Redbook, it only move fields from one data queue to the other and doesn't do any processing.

The best times I have observed from the view-controller side is less than 100 milliseconds both to send the instruction to the model running on the 170 and to received the response from the model. That is under 100 ms (70-80 actually) each way. Not bad.

One of the console Java applications (see the code for this blog) shows the time in milliseconds it takes to send the message to the data queue and then to receive the result from the demonstration server. Test it yourself!

You can expect better performance in a local area network than over the Internet. However the performance will drop when the model actually does something, has to compete for resource, etc.

Posted by Bill Blalock on March 8, 2007 at 9:30 PM | Comments (0)

March 4, 2007

Pitfalls of implementing MVC in a Java desktop with calls to System i programs

The MVC design pattern separates business logic and the user interface. For the System i a common way to separate the business logic is to put it into programs that are called by the user interface programs -- all of which reside on the System i. Most non System i clients can't directly call System i programs but Java can!. Java can execute (call) programs on the System i in two different way. Both of these ways have pitfalls for Java, Eclipse RCP or other desktop clients.

A side note ....

Actually I am already here. I hope to learn more about WDSc 7 amoung other things. Maybe even meet some of the developers.

Back to our regualarly schedule program .....

The usual way to isolate the data and business logic from the display is by coding it in programs, particularly service programs. This is not very useful to clients which aren't running on the System i. Non System i clients require a layer between the client and the programs which implement the business model. For example, web pages require the CGI layer between the HTML (web browser) and the service program.

Java can directly execute the programs which implement the business model, but not very well or very easily. The Java toolbox provides classes to execute commands and programs as well as a flavor of XML to simplify executing programs.

It most cases it takes too long to call a command or program through the Java toolbox each time the application interacts with the System i. The user won't be happy. Execution of these classes and methods has a lot of over head. The overhead is mostly on the System i which has to do all the work to prepare for the command or program, then execute it.

I am not criticizing the Java toolkit. These classes and methods are incredibly useful, and extend the capabilities of Java both on the System i and on clients connecting with a System i. But it is my experience they take too long when called each time the user does the equivalent of pressing Enter.

Java Native Interface (JNI) is a much more efficient method for Java to run programs that implement the business model. The Java JNI classes running on the System i execute programs natively. These programs can be business models, APIs, anything that could be called with CL, C or RPG! With JNI a Java client can have comparable performance to ILE in calling sub procedures, service programs, APIs, etc.

To access these JNI classes from the desktop client (or other platform) you have to use Remote Method Invocation (RMI).

Step by step it looks like this:
- the desktop application calls a local look-a-like method whose matching method is on the System i running as a RMI
- the network layer communicates with the matching RMI running on the System i
- the RMI natively uses JNI to call the business logic program or whatever written in RPG, C, CL, etc.
- the business logic program or whatever does its work and returns its results
- through JNI the results get back into the RMI program
- the network layer communicates the results to the look-alike method on the desktop
- the desktop application gets the results from the method

From the application side it is pretty simple:

results = MyRmiClass.foobar(param1, param2);

The above steps are hidden from the application programmer. The devil is in getting to the point where you are ready to write that line.

This is really a very efficient and robust method for Java on a remote system to execute programs on the System i. Really! "Remote system" can be a GUI client, another System i, a peer-to-peer application running on any platform which supports Java. Cool stuff.

And it is almost as complicated as it sounds -- but worth it if you have a Java something which needs fast and robust access to System i programs.

In summary, Java has two ways to call System i programs. The Java tool kit classes to call System i programs and commands are easy to use. For implementing MVC, they are too slow. The other way, JNI/RMI, is fast enough but is not easy to use. There is a better way.

Posted by Bill Blalock on March 4, 2007 at 9:00 PM | Comments (0)

February 28, 2007

A quick review of MVC

MVC stands for Model-View-Controller. Desktop clients for the System i should be implemented with the MVC design pattern and not use the System i simply as a data base server.

Wikipedia defines MVC as

"an architectural pattern used in software engineering. In complex computer applications that present lots of data to the user, one often wishes to separate data (model) and user interface (view) concerns, so that changes to the user interface do not impact the data handling, and that the data can be reorganized without changing the user interface. The model-view-controller solves this problem by decoupling data access and business logic from data presentation and user interaction....

Model --
The domain-specific representation of the information on which the application operates. The model is another name for the domain layer. Domain logic adds meaning to raw data (e.g., calculating if today is the user's birthday, or the totals, taxes and shipping charges for shopping cart items).

View --
Renders the model into a form suitable for interaction, typically a user interface element.

Controller --
Processes and responds to events, typically user actions, and may invoke changes on the model. "

The Model runs on the System i, it is the business logic and data access of the applications. The Model, business logic, is easily kept separate from the presentation (view and controller).

The desktop client is the View-Controller. It displays the data and interacts with the user.

A View is not easily separated from the Controller. Consider two common views, a web browser and a 5250 display. The same Controller can't operate on both Views. The View part is defined by the physical and logical display. The Controller has to interact with the View. The Controller and the View need to "know" about one another. But both pairs of View-Controllers could work with the same Model.

Typically each View has its own Controller. The Controllers for separate Views may share logic and components but they are differentiated by the requirements of the physical view that they control, process events of and respond to.

Sometimes the View and Controller must be implemented separately. That is the case with a web browser. The browser is the view. The Controller can be a CGI program or web services on the server which generates the HTML for the browser and responds to the POST and GET from the browser.

In other cases, such as a 5250 display program, it makes sense to implement the view and controller together. We will combine the view and controller with the Eclipse RCP client.

It is easier to get and manipulate the data within the Eclipse RCP client using SQL or record level access of the Java toolbox. It is also easier to write a 5250 program which does its own data base IO and handles its own data base logic. Think carefully before taking the easy way.

Posted by Bill Blalock on February 28, 2007 at 8:20 PM | Comments (0)