RPG Talks To Watson
September 27, 2016 Paul Tuohy
Note: The code accompanying this article is available for download here.
Yes, RPG can talk to Watson. No special software required, nothing to install, nothing to configure. You just need to be on V7R1, have the ability to use embedded SQL and write just a few lines of code–none of which are complicated. To see how it works, all you have to do is copy/paste the display file and RPG code in this article, compile and call.
On the off chance that you don’t know what Watson is, Watson is the IBM computer that, in 2011, competed on the U.S. quiz show Jeopardy! against former winners Brad Rutter and Ken Jennings. Watson won by a mile.
Wikipedia describes Watson as “a question answering (QA) computing system that IBM built to apply advanced natural language processing, information retrieval, knowledge representation, automated reasoning, and machine learning technologies to the field of open domain question answering.” Since 2011, Watson has gone on to be used in multiple fields, including cancer research.
But, for this example, I am going to use one of Watson’s simpler functions: language translation. The example is a green-screen program that allows me to identify a from and to language and some text, call Watson to do the translation, and display the translation on the screen.
I am not recommending that your program should be green screen (quite the opposite), but I don’t want anyone to get the impression that there is any special web configuration, or anything like that, involved in this process. So I decided that the program should be “old school”.
Let’s look at the program first and, later on in the article, I will explain where I dug out the information on how to connect to Watson.
The Display File
The display file (TOWATSOND) is very simple. It consists of one format with three input fields and two output fields. FROMLANG and TOLANG allow entry of a code to identify the From and To languages, FROMTEXT is the text we want to translate, TOTEXT is the translated text and SQLCODEO is the value of SQLCODE (in case there is an error). F3 is used to exit the program.
A INDARA A DSPSIZ(24 80 *DS3) A CF03(03) A R DATAR A 1 27'Watson Translator' A COLOR(BLU) A 3 1'Language (1=English, 2=Spanish, 3=- A French, 4=Italian, 5=Portugese)' A 4 1'From:' A FROMLANG 1 0B 4 7VALUES(1 2 3 4 5) A EDTCDE(X) A 4 12'To:' A TOLANG 1 0B 4 16VALUES(1 2 3 4 5) A EDTCDE(X) A 6 1'From Text:' A FROMTEXT 320A B 7 1CHECK(LC) A 12 1'To Text' A TOTEXT 320A O 13 1 A 18 1'SQLCode:' A SQLCODEO 5 0O 18 10 A 23 2'F3=Exit' A COLOR(BLU)
The language choices are English, Spanish, French, Italian, and Portuguese. Watson also allows translation to/from Arabic, but I did not feel up to the challenge of handling the CCSIDs!
The logic of the program (TOWATSON) is also very simple. There are a few small points worth noting. Please refer to the following callouts in the code below.
(A) The data structure defines an array of language codes. The codes correspond to the number entered for the from/to languages on the screen (1 = English (en), 2 = Spanish (es) etc.).
(B) The program loops through displaying the screen until F3 is pressed.
(C) On every iteration of the loop, the program calls the transLate_Text() subprocedure, passing parameters for the from language code, to language code, from text and to text.
ctl-opt option(*srcStmt: *noDebugIO) dftactGrp(*no); dcl-F toWatsonD workstn(*ext) usage(*input: *output) indDs(WSI); dcl-Ds WSI qualified; F3Exit ind pos(3); end-Ds; (A) dcl-Ds *n; *n char(10) inz('enesfritpt'); lang char(2) dim(5) pos(1); end-Ds; exec SQL set option naming = *SQL; exfmt dataR; (B) dow not WSI.F3Exit; (C) transLate_Text( lang(fromLang) : lang(toLang) : fromText : toText); SQLCodeO = SQLCODE; exfmt dataR; endDo; *inLR = *on;
The transLate_Text() Subprocedure
transLate_Text() is coded directly after the mainline above. transLate_Text() uses two SQL functions (URLENCODE and HTTPGETCLOB) to do all the work. URLENCODE and HTTPGETCLOB are user-defined functions that come with the system: you will find them in the SYSTOOLS library.
The main points to note in transLate_Text() are as follows. Please refer to the callouts in the code:
(A) The HTTPGETCLOB function will return a CLOB. RPG does not recognize the CLOB data type so we define textBack as a variable with an SQL type of CLOB. When the program is compiled, this definition will result in a data structure with two sub fields: textBack-Len (which will contain the length of data returned) and textBack_Data (which will contain the data).
(B) URLENCODE is called to encode the entered text. Encoding will translate any special characters that might cause problems (like & or <) to their coded equivalent.
(C) Construct the URL to make a REST call to Watson to do the translation (I will explain the construct of the Watson URL in a moment)
(D) Use HTTPGETCLOB to make a REST call to Watson. The returned value is placed in the textBack CLOB defined earlier.
(E) If data was returned, retrieve the indicated length of data (textBack_Len) from textBack_Data.
dcl-Proc transLate_Text; dcl-Pi *n; fromLang char(2) const; tolang char(2) const; fromText char(320) const; toText char(320); end-Pi; dcl-s str1 varchar(1000); dcl-s str2 varchar(1000); (A) dcl-s textBack SQLType(CLOB: 320); if (fromLang = toLang); toText = fromText; return; endIf; str1 = %trimR(fromText); exec SQL (B) values trim(systools.urlencode(:str1, '')) into :str2; (C) str1 = 'https://watson-api-explorer.mybluemix.net/' + 'language-translation/api/v2/translate?model_id=' + fromLang + '-' + toLang + '&text=' + str2; exec SQL (D) values char(systools.httpgetclob(:str1, ''), 256) into :textBack; toText = *blanks; if (textBack_Len > 0); (E) toText = %subSt(textBack_Data: 1: textBack_Len); endIf; return; end-Proc;
Let’s be honest, that was a lot simpler than you thought it was going to be!
The Watson URL
The key to this is, of course, the Watson URL. Here is the process I went through to arrive at the required URL:
I started off by Googling for “watson language translation demo,” and taking the link for “IBM Watson Language Translator Demo,” which bought me to the page https://language-translator-demo.mybluemix.net/. Although I did not know it at the time, this was a page I would come back to in order to figure out the languages I could translate between (the model-id in the request). The drop down boxes for the from and to languages gave me the list of allowed languages (in the demo) and another quick Google gave me the two character codes for each language.
My next Google was for “Watson APIs” and from the list presented I selected Watson API Explorer, which led me to this page: https://watson-api-explorer.mybluemix.net/.
I clicked on the link for Language translation, which took me to the Language Translation page where, under Translate/Get, I clicked on the option to “Translates the input text from the source language to the target language.” This provided me with the option to enter some values and try things out. I just entered two values: the model_id (more in a moment) and the text.
The following response came back:
The part of the response we were looking for was the Request URL, which was:
This is the string I needed to construct for use with the HTTPGETCLOB function. The variable parts being the from and two languages (en-es) and the encoded text (%20 is the encoding for a space).
Of course this is just the tip of the iceberg with Watson. The URL used was one for testing purposes only. A host of services (not just for translation) are available through Bluemix, some of which are free and some chargeable.
But who knew it would be that easy for RPG to talk to Watson?
Paul Tuohy is CEO of ComCon, an iSeries consulting company, and is one of the co-founders of System i Developer, which hosts the RPG & DB2 Summit conferences. He is an award-winning speaker who also speaks regularly at COMMON conferences, and is the author of “Re-engineering RPG Legacy Applications,” “The Programmers Guide to iSeries Navigator,” and the self-study course called “iSeries Navigator for Programmers.” Send your questions or comments for Paul to Ted Holt via the IT Jungle Contact page.
All the images for this article seem to have disappeared.
Great and very useful article! as usual from Paul.
To notice: The Watson API URL has changed a little bit: it is not anymore
Then at the end we should have (in the RPG code):
str1 = ‘https://watson-api-explorer.mybluemix.net/’ +
fromLang + ‘-‘ + toLang +
The length of data returned is 1077952576???
As someone posted on another forum, dec(1077952576) is hex(40404040), i.e., blank.
This means that there was no meaningful response from the service.
In my own test, I got TEXTBACK_LEN = 1077952576 and TEXTBACK_DATA = *blanks
Looking up at the job log, found this.
Message ID . . . . . . : CPF503E
Message . . . . : User-defined function error on member QSQPTABL.
Cause . . . . . : An error occurred while invoking user-defined function
HTTPGETCLOB in library SYSTOOLS. The error occurred while invoking the
associated external program or service program B2RESTUDF: in library
SYSTOOLS.D, program entry point or external name
com.ibm.db2.rest.DB2UDFWrapper.httpGetClob, specific name HTTPG00005. The
error occurred on member QSQPTABL file QSQPTABL in library QSYS2. The error
code is 1. The error codes and their meanings follow:
1 — The external program or service program returned SQLSTATE 38000. The
text message returned from the program is:
SYSTOOLS.HTTPGETCLOBHTTPG00005Premature end of file. .
Researching this online, I wasn’t able to find a meaningful description of what this error might signal. Apparently SQLSTATE 38000 is a generic state, and the key to the error must lie in the exception description “Premature end of file”. Obviously, this leaves you wishing that this error handler were just a bit more verbose.
I was able to call this language translation service via HTTPAPI instead of SQL HTTP functions, so the service itself is fine. It’s either HTTPGETCLOB() function or the way we call it.
Any further ideas as to why this error may be happening?
i got the same TEXTBACK_LEN = 1077952576 error. anyone able to progress on this one? i was able to get the correct response when i pasted the STR1 value to a browser.
It returned Bonjour
Please try to change url in “str1” like below – that works for me.: “hello” to “bonjour”
str1 = ‘https://watson-api-explorer.mybluemix.net/’ +
fromLang + ‘-‘ + toLang;
This doesn’t seems to be working .In Joblog can see this error
Java stored procedure or user-defined function SYSTOOLS.HTTPGETCLOB,
specific name HTTPG00005 aborted with an exception
Length or start position is out of range for the string operation.
TEXTBACK_LEN = 1077952576
[…] RPG Talks To Watson – On the off chance that you don’t know what Watson is, Watson is the IBM computer that, in 2011. […]
the api don´t functioning anymore