Upgrade Slim framework from 3 to 4

5 January 2020

Kwai

Kwai-api is an API for managing a sportsclub website. Instead of using a CMS like Drupal, I’ve decided to write this by composing Kwai-api with libraries and frameworks because in the past I struggled with upgrading CMS systems from one major version to another.

One of the frameworks used is Slim. Slim describes itself as : “Slim is a PHP micro framework that helps you quickly write simple yet powerful web applications and APIs.”. On August 2019, a new major version of Slim was released: 4.0.0.

Upgrading

Slim 4 has some breaking changes. All of them are listed on the Upgrade guide.

Container

Slim 4 has no built-in Container class anymore. A separate Container library which implements the PSR-11 - Container interface is needed now. No problem, that’s the way I like to work: select a library that meets the requirements and use it. I already use some libraries from the The PHP League, so I selected the Container package. Instead of using the constructor of App, AppFactory is used to create the Slim Application instance.

$container = new Container();
$container->defaultToShared();
AppFactory::setContainer($container);

$application = AppFactory::create();

Callbacks are used to create objects that are stored in the container. The current Container instance was always passed to this function. With the PHP League implementation this must be set explicitly by calling addArgument:

$container->add('template', function ($c) {
    $dir = $c->get('settings')['template_dir']
    return new TemplateEngine($dir);
})->addArgument($container);

AppFactory is introduced to decouple the PSR-7 implementation. I installed Slim PSR-7.

Kwai API uses an invokable class to execute actions. Slim passes the Container instance in the constructor. The container interface must be changed from Interop\Container\ContainerInterface to Psr\Container\ContainerInterface.

Changes to Routing components

To avoid a big PHP file with a lot of routing information, the first part of the Kwai api points to a PHP file. In this PHP file a routing group is created. In Slim 3 the Application instance was passed to the group function and the $this variable was bound to the Application instance. In Slim 4 this is changed to an instance of RouteCollectorProxy and $this is now bound to the Container instance. This is logical, because this is the same behaviour as the routing closure.

So this code :

$app = \Core\Clubman::getApplication();
$app->group('/news', function () {
    $this->get('/stories', \REST\News\Actions\BrowseStoryAction::class)
        ->setName('news.browse')
    ;
    ...
});

is changed to:

use Slim\Routing\RouteCollectorProxy;

$app = \Core\Clubman::getApplication();

$app->group('/news', function (RouteCollectorProxy $group) {
    $group->get('/stories', \REST\News\Actions\BrowseStoryAction::class)
        ->setName('news.browse')
    ;
    ...
});

I also had to set a basepath after creating the Application instance.

$application->setBasePath($basePath);

This wasn’t necessary in Slim 3. Because there are also other directories containing API’s, I changed the getApplication method and passes the basepath as argument. '/api' is the default. For judo specific api’s, '/api/sport/judo' is passed.

Changes to Middleware

Our middleware was still using the original interface:

function (
    ServerRequestInterface $request,
    ResponseInterface $response,
    callable $next
) : ResponseInterface

but the final standard has changed to:

use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\ServerRequestInterface;
use Psr\Http\Server\RequestHandlerInterface;

interface MiddlewareInterface
{
    public function process(
        ServerRequestInterface $request,
        RequestHandlerInterface $handler
    ) : ResponseInterface;
}

So each middleware must now implement MiddlewareInterface.

2010 - 2019 - A decade overview

19 December 2019

GLUEscript

In 2010 I was still working on GLUEscript. GLUEscript was the successor of wxJavaScript. The ultimate goal of GLUEscript was to create a JavaScript engine that turned JavaScript into a general purpose language. In 2009 node.js was released, which proved that using JavaScript on the server was a great idea. But the success of node, was also the reason why my interest into GLUEscript started to fade away… Node uses the V8 engine (Chrome), which was open sourced in 2008, while GLUEscript was using SpiderMonkey (Firefox). The last release of GLUEscript was on 27 January 2012.

POCO

While working on GLUEscript, I started to learn POCO. I was writing a MongoDB module for GLUEscript, so I thought that maybe POCO could use a MongoDB package. In the meanwhile I also worked on a JSON package. On 16 September 2013 the MongoDB package was added to POCO 1.5.2.

Redis

MongoDB was the first NoSQL database I got to know. The next one that got my attention was Redis. On 10 November 2017 a Redis package was released with POCO 1.8.0.

MQWeb

Another reason that my interest in GLUescript was fading, was MQWeb. In 2010, MQWeb started as an opensource project to prove that my idea “use a REST api to manage WebSphere MQ” worked. I still don’t know why this idea was thwarted, but finally in 2014 I was allowed to work on it during office hours. MQWeb was also my first project on Github.

Zumuta

Zumuta is my personal website/blog. Started in 2013 with the idea to write more about open source, but I don’t have enough time for it … One advantage of this work: I learned to generate static websites. Zumuta and MQWeb uses Jekyll.

Websites

In the meantime, I was also a webmaster for some clubs and associations. I used Xoops, Drupal, Joomla. These content management systems learned me that it was always difficult to upgrade to a next version when you have changed some code or integrated some other modules. When I volunteered to manage a website and club management system for a judo club I decided to write my own system. A like-minded colleague (thanks Bart!) was already doing this for a volleyball club. This was the start of Kwai. With Kwai, I now learn to write single page applications with Vue and REST api’s with PHP.

At work

In 2019, after 15 years, I left the MQ team and joined the network team to help them to automate recurring and manual tasks: Python, Flask, Ansible, … Again time to learn a lot! I don’t know what the future of MQWeb will be, but I saw that IBM also started to implement my idea …

One thing that I learned from contributing to open source: I’m still relevant because I continue to learn new technologies.

MQ Statistics with triggers and MQWeb

25 January 2019

What?

IBM MQ can generate statistics for queues, channels, … To be able to do some capacity management, these statistics will be transferred to ELK (Elasticsearch, Logstash & Kibana). This solution will use two Python scripts to get statistics in JSON format from MQWeb and saves the statistics to a file. With filebeat the generated files are collected and transferred to ELK.

How?

Enable IBM MQ Statistics

IBM MQ puts statistic messages on the SYSTEM.ADMIN.STATISTICS.QUEUE. This is done when the queuemanager property STATQ has value ON or when the queue property STATQ has value ON. By default the statistics are generated every 1800 seconds. The interval can be changed by setting the STATINT property of the queuemanager.

It’s also possible to force the generation of the statistics by issuing the following command on a queuemanager:

RESET QMGR TYPE(STATISTICS)

Gather statistics from a queue

Statistic messages are messages with format MQADMIN. The message payload is a collection of PCF structures. MQWeb has an API to get messages from a queue and translate MQADMIN messages into a readable JSON structure. The mqstats.py script will use MQWeb to get these messages and output them to stdout or to a file. The script ends when there are no more messages to process.

Triggering

The mqstats.py script must run when messages are available on the queue. To avoid the creation of a new daemon process, the IBM MQ triggering system is used. IBM MQ can trigger the script whenever the queue depth of the queue changes from 0 to 1. Because IBM MQ passes an MQTMC2 structure to the triggered process, another script mqstatstrigger.py is written. This script will translate the MQTMC2 structure into commandline arguments for mqstats.py and execute it.

The mqstats.py script can be used directly from the commandline. Use the –help argument to show help information. When the MQ trigger doesn’t fire (because there are still messages on the queue), run the mqstats.py from the commandline to process the remaining messages.

Follow these steps to setup IBM MQ triggering:

Initiation queue

When a trigger event occurs, the queuemanager puts a trigger message on a initiation queue. This trigger message will be retrieved by the trigger monitor.

DEFINE QL(MQ.STATISTICS.TRIGGER.Q1) +
DESCR('Initiation Queue For Collecting Statistics')

Define a process

The process defines what the trigger monitor must execute when the trigger is fired.

DEFINE PROCESS(MQ.STATS) +
APPLICID('python mqstatstrigger.py') +
USERDATA('output=/var/elk') +
DESCR('ELK statistics script')

The USERDATA value is used by the mqstatstrigger.py script to pass arguments to mqstats.py

Alter the local queue

The queue that is responsible for the trigger must be associated with the initiation queue and the process:

ALTER QL(SYSTEM.ADMIN.STATISTICS.QUEUE) +
INITQ(MQ.STATISTICS.TRIGGER.Q1)
PROCESS(MQ.STATS) +
TRIGTYPE(FIRST)

Define a service

The runmqtrm program is the IBM MQ trigger monitor. It must run in background. When a service is used, it can be controlled by the queuemanager by setting the CONTROL property to QMGR. This way the service will automatically start and stops when a queuemanager is started or stopped.

DEFINE SERVICE(MQ.STATS.TRIGMON) +
CONTROL(QMGR) +
STARTCMD('+MQ_INSTALL_PATH+bin/runmqtrm') +
STARTARG('-m +QMNAME+ -q MQ.STATISTICS.TRIGGER.Q1') +
STOPCMD('+MQ_INSTALL_PATH+bin/amqsstop') +
STOPARG('-m +QMNAME+ -p +MQ_SERVER_PID+') +
STDOUT('/var/elk/mqstats.stdout') +
STDERR('/var/elk/mqstats.stderr') +
DESCR('Trigger Monitor for ELK Statistics')

MQWebApp

13 January 2016

The web application that was part of MQWeb is moved to a separate repository MQWebApp and removed from the current development branch of MQWeb. The web application will be a totally rewrite as a Single Page Application for WebSphere MQ using Vue.js and UIKit.

This MQWeb App will only be supported from MQWeb 0.1.0. and above.

Screenshot

Poco Redis

21 November 2015

Today the development of the Poco Redis module is merged into the Poco development branch.

Some examples:

###Setting a key/value with add

Array command;
command.add("SET").add("mykey").add("Hello");

// A set responds with a simple OK string
try
{
  std::string result = redis.execute<std::string>(command);
}
catch(RedisException &e)
{
  ...
}

###Setting a key/value with «

Array command;
command << "SET" << "mykey" << "Hello";

// A set responds with a simple OK string
try
{
  std::string result = redis.execute<std::string>(command);
}
catch(RedisException &e)
{
  ...
}

###Setting a key/value with Command class

Command set = Command::set("mykey", "Hello");

// A set responds with a simple OK string
try
{
  std::string result = redis.execute<std::string>(set);
}
catch(RedisException &e)
{
  ...
}