Guru: Web Concepts For The RPG Developer, Part 3

July 22, 2024 Chris Ringer

Greetings everyone. Articles part one and two were both an introduction on how to build components in an HTTP request. In part three, we will begin to connect the dots and discuss how to asymmetrically sign a simple string. If you ever need to send a secure HTTP request to a government agency or financial institution, you likely will authenticate with a signed token. So, here we go!

Base64 Take Two

In part one, the SQL scalar function BASE64_ENCODE embedded in RPG converted a string to base64. This technique will cover most use cases, but what if you need to convert an entire IFS stream file to base64? Yes, you could read it into an RPG variable, convert the variable to base64 then write the base64 back to the IFS but this feels cumbersome. And RPG has a max varchar size of 16 MB. The base64 encoding command in openssl will convert IFS files, even files much larger than 16 MB.

And before you start using QSHELL for this article, we want to make sure any IFS stream files we create in QSHELL are not encoded as EBCDIC (your job CCSID). We want them all to be plain ASCII (CCSID 819). Otherwise, some character conversions may occasionally fail. Openssl may create some stream files as UTF-8 ASCII (CCSID 1208) and that’s fine too. Run this command in your job to set the QIBM_CCSID environment variable.

ADDENVVAR ENVVAR(QIBM_CCSID) VALUE(819) REPLACE(*YES) 

Figure 1

Now, on to QSHELL. For a baseline, this is my openssl version. Start STRQSH and run this command to see your version:

openssl version

Figure 2

For this example, we’re going to convert a PDF stored on the IFS to base64. Create a test folder in your home directory by running these commands individually (without the line numbers).

01 cd 
02 pwd
03 mkdir part3_1
04 cd part3_1

Line 01: A change directory without options changes to your home directory.

Line 02: Displays the present working directory (your home directory).

Line 03: Makes a new sub-directory.

Line 04: Changes the current directory to be part3_1.

Figure 3

Then copy or upload any file (a PDF in my case) to this new directory and list the attributes.

ls -lS

The ls command lists the contents of a directory. The -l (dash ell) option is the long format which includes the file size, creation date and file name in the last three columns. Capital S means show the file CCSID, in column 1 (figure 4).

Figure 4

Run this command to encode the PDF into base64.

openssl base64 -e -A -in "my_doc_ver1.pdf" -out "my_doc_ver1_pdf_base64.txt"

The base64 command line options are:

-e : encode to base64

-A : output without any line feeds. Openssl likes to make the output human friendly on the screen but we don’t want or need line feeds.

-in : the name of the input file. The double quotes are optional but I like to include them in case the file name contains embedded spaces.

-out : the name of the output file to contain the base64 text.

And run this command to decode the base64 back to a PDF.

openssl base64 -d -A -in "my_doc_ver1_pdf_base64.txt" -out "my_doc_ver2.pdf"

The new base64 command line option here is:

-d : decode the base64 file.

Ok, let’s see what we have now. List the directory contents.

ls -lS

Figure 5

The version 1 and version 2 PDFs have the exact same file size (150575 bytes), but how do we verify that they contain the exact same content? We simply compare the message digests of the two files (figure 6). Notice that the sha256 output is human readable hex, not raw binary, and the hex value is 64 bytes long (512 bits) whereas sha256 can also produce 32 bytes (256 bits) of binary output. This distinction will be important later.

openssl dgst -sha256 my_doc_ver*.pdf

Figure 6

To view all the base64 command options enter:

openssl base64 -help

Asymmetric Signatures

For machine to machine (M2M) communications over HTTP, there is no person in a chair entering a password and a MFA pin number. An M2M HTTP request could still contain a client ID and password but that could be intercepted by a bad actor and cause all sorts of problems.

Moreover, you may be familiar with asymmetric encryption where clients (like browsers) use a public key to encrypt the HTTP request and the server uses the corresponding secret private key to decrypt the HTTP request (in the SSL/TSL Handshake). Well, an asymmetric signature is the opposite. Only the client (you) posseses the private key and freely shares the corresponding public key with other parties.

You asymmetrically sign a file (or a string of text) using the private key and send that result to another party via HTTP. The other party will verify the signature using the public key. If the signature verifies as valid then the recipient knows only the client possessing the private key could have signed the string. This is the authentication. The private and public key pair are linked by two prime numbers thousands of digits long and some very sophisticated math that I will never pretend to understand. Again, only the client (you) posseses the private key.

That’s enough theory. Next we will generate a private key and the corresponding public key. Run the STRQSH command then run these three commands individually.

01 cd
02 mkdir part3_2 && cd part3_2
03 openssl genrsa -out my_private_key.pem 2048

Line 01: Change to your home directory.

Line 02: Makes a new sub-directory and changes to it. The && allows you to chain commands together. If the previous command ran successfully, only then will the next command run.

Line 03: Generates a new private key file using the genrsa command. RSA is an encryption algorithm using asymmetric (private/public) keys. Your favorite browser uses RSA. Don’t worry. We don’t need to understand how it works to use it!

Figure 7

A file named my_private_key.pem was generated with a key length 2048 bits (256 bytes). The PEM format includes a header and footer, and several key lines 64 characters long in base64 format ending with line feeds (Figure 8).

Figure 8

And run these two commands to generate the corresponding public key and list the file details (Figure 9).

01 openssl rsa -in my_private_key.pem -outform PEM -pubout -out my_public_key.pem
02 ls -lS

Line 01: Reads in an RSA private key and writes out the corresponding public key in PEM format.

Line 02: List the directory contents (Figure 9).

Figure 9

Alright, now we’re getting somewhere! We’re ready to digitally sign any stream file. Let’s create one.

01 printf "%s" "Hello World" > HelloWorld.txt
02 cat HelloWorld.txt  

Line 01: printf writes the string “Hello World” to the screen but the > redirects the output into the new text file. Notice our file is now 11 bytes long (Figure 10) as expected. Echo can do the same thing but it likes to add a line feed to the output.

Line 02: Reads the new text file and displays on the screen, just to verify.

Figure 10

Run the following commands as one long command to sign the file and convert the signed message digest to base64 (Figure 11).

01 openssl dgst -sha256 -binary -sign "my_private_key.pem"
02 -out "HelloWorld_signed.bin" "HelloWorld.txt" && 
03 openssl base64 -e -A -in "HelloWorld_signed.bin"  
04 | tr '+/' '-_' > "HelloWorld_signed_base64.txt" 

I know, that’s a lot, so let’s break it all down.

Lines 01 – 02: These lines create a 256 byte binary message digest and binary here means non-text, a long stream of bits. This message digest is created based on file HelloWorld.txt and signed using the private key file. The signed message digest file is named HelloWorld_signed.bin. The trailing && means if the message digest was created successfully, run the next command. Notice our binary message digest file is 256 bytes in size (Figure 11). And to be clear, we are actually signing the message digest, not the text file.

Line 03: Because web API endpoints typically expect to receive text data, we have to convert our binary message digest file to base64. The option -e means encode, -A means do not insert line feeds into the output.

Line 04: The output of the base64 command is piped into the translate command, which is similar to the RPG %XLate function. To make our base64 URL safe, all ‘+/’ characters are translated into ‘-_’ respectively. Notice our base64 file size grew 34 percent from the message digest file size.

Figure 11

At this point, you would transmit both the HelloWorld.txt contents and the base64 signature over HTTPS to a web API endpoint which would verify the signature using the public key. Let’s simulate that verification now. Translate the URL safe characters in the base64 back to the original values and decode the base64 into a binary message digest file.

01 cat HelloWorld_signed_base64.txt | tr '-' '+' | tr '_' '/' > HelloWorld_signed_base64_2.txt
02 openssl base64 -d -A -in "HelloWorld_signed_base64_2.txt" -out "HelloWorld_signed2.bin" 
03 openssl dgst -sha256 HelloWorld_signed*.bin

Line 01: Translates any base64 characters ‘-_’ back to ‘+/’ respectively into a new text file.

Line  02: Decodes the base64 into a new sha256 binary message digest with size of 256 bytes (figure 12).

Line 03: Compares the before and after message digests for the two binary files. Should be the same.

Figure 12

The binary signature can be verified against a file or an HTTP payload. Both of the following commands accomplish this.

01 openssl dgst -sha256 -verify my_public_key.pem -signature HelloWorld_signed2.bin HelloWorld.txt  
02 printf "%s" "Hello World" | openssl dgst -sha256 -verify my_public_key.pem -signature HelloWorld_signed2.bin

Line 01: Verify the signature (the message digest) with the public key against the HelloWorld.txt file. “Verified OK” is self-explanatory (figure 13). Only someone possessing the private key could have signed the message digest. And the calculated message digest is correct for the HelloWorld.txt expected value so the data was not tampered with (no “Level Checks”).

Line 02: Same as Line 01 but the string “Hello World” is piped into the command.

Figure 13

Now let’s purposely force the signature to fail verification. Run these next two commands.

01 printf "%s" "Hello Wayne" | openssl dgst -sha256 -verify my_public_key.pem -signature HelloWorld_signed2.bin
02 printf "%s" "Hello World" | openssl dgst -sha256 -verify other_public_key.pem -signature HelloWorld_signed2.bin

Line 01: Attempts to verify a string not matching the expected message digest. Result is “Verification failure” (figure 14).

Line 02: Used the wrong public key, so this verification fails, too. If the client uses the wrong private key then verification will also fail.

Figure 14

Outro

I attended a session called “Hidden Gems” at the Fort Worth POWERUp conference. The openssl toolkit was not mentioned but to me it falls into that category because it allows us to access cryptographic services through simple commands instead of the complex IBM APIs. And this article was a longer than normal for me but it contains lots of explanation and not much code. I was able to copy and run all the commands in about five minutes. If you’ve read all three articles in this series, then you will be prepared for the next article on creating JSON web tokens. Happy coding!

 Chris Ringer began coding RPG programs in 1989, and after a recent unexpected but valuable detour to C# is happy to be back in the IBM world. In his spare time he enjoys cycling and running – and taking the family dog Eddie for walks.

RELATED STORIES

Guru: Web Concepts For The RPG Developer, Part 2

Guru: Web Concepts For The RPG Developer, Part 1

Guru: The PHP Path To Victory, Part 1

PHP Crash Course For RPG Developers

Installing PHP on Your Laptop

Debugging Server Jobs In Green Screen