Phone:

+420 257 288 211

Process and service modelling for the General Customs Directorate of the Czech Republic

At first glance, numerical data lists are simple tables of information containing a series of codes, their definitions, dates of validity, and other relevant attributes. In actuality, however, the situation is quite the opposite. These relatively simple-looking tables are a key component of the information system as a whole. If these numeric tables contain any errors, even the simplest application does not function as it should – data is processed incorrectly and the output contains invalid information.

Client:

Generální ředitelství cel

Client’s initial situation

The Customs Administration of the Czech Republic is the national authority that has long been successfully developing the information system it uses to support the processes associated with controlling and collecting customs duties and taxes. Various numerical tables are one of the key components of the entire system and, without these tables, it would be very difficult to manage the relevant processes and process the required data. In many cases, these are not just simple tables containing a list of codes and their definitions, but rather comprise a complete interlinked system where the values contained in the numerical tables are very closely interrelated.

The client was using a system that used outdated technology (Visual FoxPro 9) for managing numerical tables and the data files were stored locally. This type of system no longer met operational requirements and the solution was not adequate from the perspective of security or operations overall. As a result of this situation, a decision was made to launch a project aimed at preparing a new system built using new technologies and meeting the most current security standards.

Concept and technology

For a system that processes 500 numerical tables, with data that is provided from various sources and in different formats and changes on a daily basis, it was necessary to propose the system architecture in a way that ensures that the data reaches the users as soon as possible after its transformation. The required process thus corresponds to the standard ETL (Extract, Transform, and Load) process, and had to be supported by a modelling tool. After some consideration, we opted to use the Windows Workflow Foundation (WF) technology from the Microsoft family.

A graphical design module based on Windows Forms and Windows Workflow Foundation was created for working with the system and for creating complex commands. Thanks to this interface, it is easy to build a custom data process. A process created in this manner may be initiated either automatically or manually from the application, thus making it possible to capture the process in the common XAML format. This particular format is suitable not only for repetitive graphical modifications. The actual modelling of the command is performed through adding and combining various graphics elements in basic programming constructs, such as sequences, conditions, and cycles. The behaviour of the command is adjusted through changing the parameter values of the elements using either literals, or expressions and functions.
Users are not presented with only the graphical view of the modelled command; its textual representation is also provided for the purpose of clarity. This helps the user to navigate through the created data flow and provides the option for direct links between the graphical design and the text representation.

Custom solution

The graphics interface is only one part of the overall solution, which also includes the following:

  • Input interface: the part of the system that processes incoming data in the common XML format, and is accessible through web services;
  • Core system: the main part of the system responsible for the actual completion of the processing; and
  • Output interface: the part of the system responsible for generating the data for the users and sending the required information.

The processing of data in the system designed in the aforementioned way is thus a straight-through process without any needless delays unless they are defined by the user. The processing rule, according to which the assigned flow for performing the specified operation is initiated in the system, is applied to the data at the input interface. It is possible to test the data flow prior to publishing it.

The user may define various combinations of conditions at the input data level, which, if they are met, will perform the following:

  • Transfer and convert the input data from the data queue into SQL tables on the SQL application server; and
  • Initiate the required workflow for processing the input data.

When defining the rules, it is necessary to define the actual tasks (where a “task” is understood to be an activity that triggers one or more consecutive workflows), which also contain the attributes that control the tasks.

In order to provide support for working with the data it was also necessary to integrate certain other functions, such as data import and export, data modification, working with variables, table comparisons, sending of e-mail messages, and others, which, when combined with the basic functions, such as cycles, groupings, and conditions, provide an integrated tool for supporting modelling. The following activities are only individual components of the entire system, which may use predefined functions within the context of the modelling process:

  • Text functions, which work with text strings;
  • Conversion functions, which convert data values;
  • Mathematical functions, which work with numbers;
  • Date and time functions, which work with calendars and time; and
  • Others, which provide supplemental operations for values or variables.

The format for the functions is, for the most part, based on the T-SQL language. In those cases where it was not possible to use this standard, the PowerShell format was used.

One of the most complicated areas for which a solution had to be provided in this system consists of linking numeric code tables in situations when the same codes from various sources have different validity periods. The system is also capable of maintaining and publishing multiple versions of the numerical code tables. As not all users are able to modify systems for updated versions of the code tables and may not need an updated version, the current version may be sufficient for them.

Evaluation and benefits

Within the context of building data flows, the user will surely value not only the ability to graphically prepare the entire processing flow for the input data, but also the fact that each of the components added to the flow has an integrated validation function, which will inform the user of an errors in the recorded expression and/or parameters. The optimisation of the reusable function blocks built using individual activities or even entire composites, which may be called by one another with the use of transferred parameters, provides yet another bonus greatly valued by the users the very first time they work with the system.