In April’s System i NEWS UK, I wrote a piece about multi-tiered RPG applications and using RPGLE service programs to provide the database and business logic layer for any front end to use, be it browser-based or green screen. This multi-tiered approach to development came about through a requirement to provide a system where we could write the business logic once, but provide more than one method for display.

For example, and this is a real life example, some of our outlets needed to check information online from their till points but they didn’t have access to a separate green screen display. The check had to show current, up-to-date and meaningful data, so we couldn’t go to a separate server that was updated on a regular basis. The search facility had to be accessible from both a green screen and from a browser-based system because head office staff should really be using the same functions as the users, not only to save our development guys some time on development, but also to make sure that any queries from the users to head office were dealt with by people with access to the same information.

So how should it be done? There was already a single search facility available on green screen, but the logic for the display screens was embedded within the business logic code (as most display logic is in old RPG programs). However, there were some data search sub-programs that weren’t explicitly ‘modules’, but had been programmed in a modular fashion. It was decided (rightly or wrongly, but I’ll discuss this later) that the best way to use these pieces of code was to convert them to sub-procedures, put them in a service program and re-write the display layer. The display layer could then be separated as I described in the earlier article.

This was my first introduction to genuine two-tier RPG and I did (and still do) think that it is the way forward for a lot of RPG programming (or at least for situations where there is a need for more than one means of display). However, the programming techniques are not the usual green screen development skills held by most RPG programmers. I have mentioned before about the lack of interest in the ILE (Integrated Language Environment) in a lot of smaller iSeries shops, and I fully understand that a lot of these shops don’t have time to learn /Free or work out how they could begin to use the ILE. In fact, it took about eight years before someone showed me embedded SQL!

However, these shops, running V5Rx boxes with legacy code, are the ones who should really be looking to pick up these ‘new’ skills. I say new, but embedded SQL has been around for far longer than my programming skills go back. /Free is over six years old, and the Integrated Language Environment has been around for over ten years. Bob Cozzi has a little to say on the subject at: www.mcpressonline.com/mc/.6b4f33c5.

Anyway, back to the matter at hand; two-tier development on the iSeries, where the business and display logic are separated, but more than one display will be required. How should it best be done? I have described a little bit of a problem and now I would like to look at some possible solutions. The first one, CGIDEV2, is less a multi-tiered solution than a solution developed to quickly put iSeries information on to the web. It appears to be primarily a toolset (now helped by a development framework called Renaissance) to quickly enable web-based access, using the Common Gateway Interface (or CGI) standard for the internet. Rather than manually build a CGI response to a request from a browser, the service program included allows existing RPGLE programs to ‘read input from a browser, generate and send the appropriate response back to the browser.’

It allows the HTML for the web page to be separated into a template file, thus keeping the display and business logic apart, which is really the core principle behind two-tier development. However, it doesn’t quite fulfill the requirements for a write-once set of business logic for multiple methods of display, as the green screen display layer and interactive business logic will need to be coded separately as CGIDEV2 programs all run in batch.
Nevertheless, this doesn’t prevent it from being a highly successful means to get your web pages up and running and, added to the fact that it is all open-source, it is a very cost-effective method without having to resort to SQL. The CGIDEV2 service program is available for download at www.easy400.net/easy400p/main.html (including all the source code, and the Renaissance framework).

The next possible solution could have been stored procedures. Stored procedures differ from the previous method in that they are programs that can be called from within SQL and other programming languages (such as RPG), thus allowing Net.Data or Java to call RPG programs and display the result sets using either the HTML in a Net.Data script, or by using Struts 2 or JSPs in the case of Java. CGIDEV2 works by accessing the HTTP APIs that read and write a web page and all programs are called in batch, whereas stored procedures are completely interactive, only being called upon request.

I call the stored procedures programs, but they are not managed by OS/400, but by the database. They used to work by using a C (or C++) compiler, which converted the stored procedure into C (or C++) routines. I am not sure if they still work like that on the System i. However, from IBM DB2 UDB version 8.2, the stored procedures are converted to native representations and stored in the database system catalogues. On our trusty System i, though, DB2 is not a separate database, but it is rather more like part of the operating system. Thinking about it, since all the stored procedures are found by using Operations Navigator within the Database tab, it is highly likely that later versions of i5/OS work in this manner (where stored procedures are converted to native representations, rather than being compiled into C routines). I would appreciate some correction on this point if I am wrong.

Capable of anything?

Anyway, these stored procedures are extremely capable database functions. They can call RPG programs, even those not written to take advantage of the ILE. They can call or be called by any high-level language program and they provide any browser-based system with a set of IN, OUT and INOUT parameters, plus the ability to produce result-sets that can be used by Java or Net.Data. So they fulfil some of the criteria we need for a multi-tiered system. Business logic can be written once, in a modular style, in RPG. The display logic can be separated into a browser application or a green screen application. However, the stored procedure is only really used by the browser system to return a result set.

Unfortunately, though, there is no real way to achieve a level of persistence to a browser. In addition to this, the stored procedure cannot really handle a queue of messages. It can pass back information, it can call RPG programs that can validate some information and update a file, it can return single values such as the amount of tax to be paid when selling goods to different countries, and it can provide SQL functions within Java or Net.Data with a result-set that can be iterated through to produce a screen of dynamic data. In today’s world of service-oriented architecture, stored procedures can be a great way to get more use from legacy applications and modules. A simple CREATE PROCEDURE statement for each RPG program you want to call, identifying the program, the language it was written in and the parameter list required for execution, is all that is required to create a stored procedure. It may take a little re-engineering of your legacy code to strip out the display logic but, overall, you will end up with code that is more reusable and less complicated.

So for true multi-tiered layering, should we use stored procedures? They are definitely very useful in allowing any variety of modular RPG to be accessed from another system, but there are issues that stored procedures can’t really handle very well. The concept of persistence is something that is difficult for a browser-based system to deal with without assigning complicated session IDs. Stored procedures are nothing more than database functions, so they don’t understand persistence. Message queue handling is also difficult to achieve. In RPG-world, we can assign indicators to fields to highlight those in error. We can use message sub-files to display the associated message text. So how can a multi-tiered system get round this?

Well, the solution that was eventually used was one that I described in my April article. Service programs were used, with individual sub-procedures for different tasks, such as loading a sub-file, validating certain sub-file options dependent upon the mode of the program, deletion of records if there were acceptable reasons, and updating of the database, provided there were no record locks. All these can be handled using a combination of RPG and embedded SQL.

Embedding the SQL within the sub-procedures allowed a similar level of control to using SQL and stored procedures. Allowing the called sub-procedure to create a timestamp gave us a level of persistence in the open cursor. This timestamp was then passed back to the display layer, which in turn passed it back when the business layer was called again. If the timestamp passed back matched the one in the business layer, then the cursor was re-used. If not, then it was closed and re-opened.

Filtering on sub-files could be achieved by validating the entered ‘filter’ key in a new sub-procedure. The validated filter could then be passed in the parameter list, a WHERE clause could be built in the RPG, and the embedded SQL could then dynamically build the data based on the filter. This allowed both ‘position to’ and character matching filters over sub-files. Message queue handling within the browser was achieved by allowing the Java-based browser display to create a unique key. The ‘validateDataInput’ sub-procedure would accept a screen full of data along with the unique key, and the name of a keyed data queue, and all validation required would be completed within the sub-procedure.

Each potential validation message would be given a unique message ID, along with pre-defined text, and these would be defined in the D specs of the sub-procedure. If the validation failed at any point, for each error message a CL program would be called with the message ID and text, and depending upon the display layer (green screen or browser), the resulting message ID and text would be either be sent to the program message queue (for green screen) or the keyed data queue for the browser display.

Different class

Within the browser display layer, a new Java class was created to take the unique key, retrieve the messages and the associated text from the data queue, and apply them to the display. This Java class had the advantage of being reusable for any new multi-tiered application. In addition, an array of 22 characters was also provided to allow the business layer to pass back indicators to the green screen display. This allowed indicators to be used to highlight fields that had failed validation and also for error message text to be presented in either a message sub-file or a separate window. (For the window option, another sub-procedure was used to display all the messages together).
This eventual solution had the advantage of having all the error message text assigned by the business logic, so a direct parallel could be maintained between the RPG display layer and the Java display layer. It meant that we had identical functionality between the green screen display and the browser display, and this enabled us to achieve our aim of one unified business procedure for different display layers. It gave complete harmonisation between the two display layers, without having to resort to a screen-scraping solution (which, to be honest, would never have been acceptable as a solution).

Obviously, this kind of solution may not be to everyone’s taste. Some might say that if you want to web-enable some services, you should simply add a new suite of programs that do what you need. We only really tend to use the two-tier solution when we really need to have a dual display layer. For applications where we only envisage web access, stored procedures work really well for us. Likewise, with green-screen work the overhead in multi-tiered development can delay projects significantly, so we stick with what we have done before.

Modularisation is the key word, though. Keep things simple to understand and provide enough documentation within the program to allow someone else to use it. Brian Meyers’ Essential Style Guide on System iNetwork.com puts it well: ‘Sometimes good style and efficient runtime performance don't mix. Wherever you face a conflict between the two, choose good style. Hard-to-read programs are hard to debug, hard to maintain, and hard to get right. Program correctness must always win out over speed. Keep in mind these admonitions from Brian Kernighan and PJ Plauger’s The Elements of Programming Style:

“Make it right before you make it faster.
Keep it right when you make it faster.
Make it clear before you make it faster.
Don't sacrifice clarity for small gains in
efficiency."