X11 still provides the basis for the Graphical User Interface of numerous Unix like systems.

While it was a userspace stack of libraries and programs, a significant chunk is now developed in kernel space and is system dependent (it's obviously machine and GPU dependent, but since it involves tight memory handling, it is very OS kernel dependent—BTW, it doesn't help that, in GPU world, kernel is used with another meaning entirely).

For the parts in user space, X11 consists of a large set of modules (184), each having its own VCS (git) repository. For a very long time, the building and installing framework was using the GNU Autotools. The conversion to the Meson build system is on the way, Autotools being generally still available alongside—but in some cases, it has been removed due to the burden of having to fix two frameworks when some change is made.

Here is a tour about different issues or notes about the shift to Meson.

  1. A list of X11 user space modules
  2. The aim of the Meson build files
  3. Autotools to Meson, step by step
    1. References
    2. The differences in a nutshell
    3. Identifying the project
      1. configure.ac
      2. meson.build
    4. Declaring the subdirs before using their variables
      1. Makefile.am
      2. meson.build
    5. Configuration: variables and options
      1. Trimming the pkg-config file
      2. xcb-proto.pc.in
      3. Adding to meson missing switches and variables
      4. meson_options.txt
    6. Setting variables
      1. meson.build
    7. Testing for xmllint
      1. src/meson.build
    8. Handling the python files and problems
      1. xcbgen/meson.build
    9. What has to be added to Autotools files
      1. Makefile.am
    10. Autoconf idiosynchrasies
  4. The C.I. facility
    1. References
    2. The Continuous Integration in short
    3. Augmenting the number of stages?
    4. Needed tools in container (packages)
    5. Adding steps
      1. Adding another building with Autotools
      2. Adding building with Meson
      3. Testing Autotools and Meson same result
      4. Verifying that Meson can work with an Autotools created tarball

A list of X11 user space modules

A list of X11 modules can be found in modular, in the module-list.txt file.

The aim of the Meson build files

Obviously, when adding Meson build files, the aim should be to provide at least what is provided by the GNU Autotools built: both the installed result and, at first, whatever flexibility is allowed to specify options with the configure.

In order to verify that the added meson configuration is providing, by default, on some system, the same thing as the existing autotools configuration, the C.I. framework of the freedesktop's gitlab service, has to be made to compare the results provided by the two ways of building. But this does not mean that the meson configuration has to mimic the autotools variables names, for example in the pkg-config generated file. How to achieve this is explained below.

Autotools to Meson, step by step

Here are some notes about the translation from Autotools directives to Meson instructions in meson.build and, if needed, meson_options.txt.

As an example, we take the Autotools configuration files from xorg/proto/xcbproto to illustrate the translation.

References

The differences in a nutshell

A Meson instructions file is called meson.build. The entry point is the meson.build file placed at the root of the module directory.

If there are subdirectories to process, a meson.build has to be written, in the corresponding subdir, for the processing. In Meson, the root file does not export variables to the children (subdirs); this is the reverse: the variables defined in a subdir meson.build are available in the root meson.build. So one has to instruct to treat the subdirs first, and to use eventually, after, in the root meson.build the variables set by the children. The instruction to inform about a subdir to be processed is subdir('some_subdir').

The syntax of Meson ressembles that of Python, but this is not exactly Python: conditionals, for example, are different. You need to refer to The Meson Build System for details.

The instructions are sequential: executed in the order they are written. So, obviously, one has to set options before building. If the predefined options are not enough for the package, options can be added with an auxiliary file: meson_options.txt (for Meson version < 1.1; meson.options is preferred from 1.1 on (but meson_options.txt seems to be still supported), holding definitions of non built-in options.

For now, meson_options.txt should be used, but it may be that, in the future, a symbolic link to meson.options should be used instead.

To write the Meson instructions, one has hence to master both the autoconf specifications, and the automake ones—I mean configure.ac and Makefile.am—and to keep in mind the defaults set by the Autotools framework that may be different from default definitions in Meson (see below).

Identifying the project

In the configure.ac, the AC_PREREQ gives the minimal version of autoconf able to process the file; in the same way, one better has to set the meson_version keyword in project() with a string describing the versions able to process the meson.build instructions (this shall be generally not an exact match but a >= conditional). This minimal version depends on the features used, indicated alongside features in the Meson reference. Be conservative: don't force the use of a "cutting edge" version hence forcing builders to update Meson and, perhaps, to have to update Python as well.

The AC_INIT holds the triplet (package, version, bug-report-address) that maps directly to some definitions in meson.build first function project('package', [language...,] version : 'version', license : 'license', meson_version : 'conditional'...); the license, though, has to be deduced from the COPYING file.
Only the first parameter (project_name) is mandatory. The second one (languages) is optional; after come the keywords definitions, in whatever order.

For the languages, one has to look at the AC_CONFIG_FILES in order to see what files are processed by the configure script (generally the pkg-config and Makefiles); this gives information about what languages are processed. In our example, the processing is for xml verification or for python processing, and these are not supported languages (for now) of the Meson building framework (see Meson project()). So we set only the project name, and the keywords for the version, license and meson_version:

configure.ac

AC_PREREQ(2.57)
AC_INIT([XCB Proto],
        1.17.0,
        [xcb@lists.freedesktop.org])

meson.build

project(
  'XCB Proto',
  version : '1.17.0',
  license : 'X11',
  meson_version : '>= 0.58.0',
)    

Declaring the subdirs before using their variables

In the root Makefile.am may be defined SUBDIRS. Generally, this will map directly to what has to be done i.e. create instructions for every such subdir. But note: specifying the subdir does not magically process things: the Meson process doesn't use Makefiles or stubs (Makefile.am or Makefile.in): one will have to write a meson.build for the subdir.

Makefile.am

SUBDIRS = src xcbgen

meson.build

subdir('src')
subdir('xcbgen')

Configuration: variables and options

Some variables, to be used by GNU Makefiles, are always defined, with default values. We show here the Autotools names and default definitions and, in regard, Meson pre-defined such variables, on the same line if the variables, having or not the same name, are supposed to designate the very same thing.
This list has to be taken as a reference, because whatever is used by the Autotools building for the module and is not a default built-in option for Meson has to be added someway.

  • Both in Autotools and in Meson, DESTDIR can be set in the environment in order to instruct the framework to put the hierarchy there and not at the root. But DESTDIR is not a settable in Meson: it has to be set in the environment;
  • The variables in Meson have defaults that are just a subdir, without, for example prefix added: this has to be added explicitely for absolute (minus DESTDIR) paths or to use relative directories, letting Meson add implicetly the prefix. In order to avoid testing, all paths, including the ones defined via options are taken as being relative to prefix, that will be prepended;
  • What has to be added is what is needed to achieve configuration and, at least, the flexibility provided by these options for the building with Autotools. Not everything has to be added by default, when it is simply not used by the Autotools building or installation;
  • Beware of datarootdir vs. datadir, in Autotools. datarootdir is the general purpose such dir for Autotools, while datadir is the corresponding name for Meson. Instead or re-introducing a datarootdir for Meson—that would have to be switched with Meson built-in datadir—we will simply, when needed, introduce another variable, whose name will say what its purpose is: for example pcconfdir for the dir where pkg-config file has to be added;
  • includedir in Autotools (GNU standard) is not the general purpose such dir: it is reserved for gcc; oldincludedir is the correct general purpose such dir, corresponding to includedir for Meson.
Gmake variable name Meson variable name Description Autotools default Meson default Commentary
prefix prefix Its value is used as a prefix to most default values of the variables listed below. /usr/local /usr/local (Unix)
C: (Windows)
Not used ``as is''—it's a prefix.
exec_prefix It is used as a prefix in default values of directory variables related to machine specific files (such as programs or libraries). ${prefix} Not used ``as is''—it's a prefix.
bindir bindir Directory for user directly executable programs. ${exec_prefix}/bin bin Installation time.
sbindir sbindir Directory for system administrators directly executable programs. ${exec_prefix}/sbin sbin Installation time.
libexecdir libexecdir Directory for programs not directly called by users. ${exec_prefix}/libexec libexec Installation time. For Autotools it is a prefix, and since these are supposed to be used by some package, the package-name is generally appended, with possibly a custom hierarchy under it.
datarootdir datadir Directory for read-only architecture-independent data. ${prefix}/share share Installation time. Beware of distinction of name. Autotools has also datadir, but with a distinct meaning. See below.
datadir Directory for idiosyncratic read-only architecture-independent data. ${datarootdir} Installation time. This is an alternate datadir, allowing for example to put such read-only architecture-independent data but shared only by some systems, elsewhere than data shared by all systems.
sysconfdir sysconfdir Directory for read-only data pertaining to a single instance. ${prefix}/etc etc Installation time.
sharedstatedir sharedstatedir Directory for architecture-independent data (autotools: modified by processes). ${prefix}/com com Installation time. Not used.
localstatedir localstatedir Directory for not architecture-independent data files, pertaining to a single instance, and modified by processes. ${prefix}/var var Installation time.
runstatedir Same nature of files as for localstatedir but that do not need to persist after process exit. ${localstatedir}/run Installation time.
includedir Directory for header files (but special case, by default, for GCC). ${prefix}/include For Autotools install, the default is generally of no use for the installation, since compilers generally don't search in the default value dir; oldincludedir should be used instead.
oldincludedir includir Directory for header files (general such dir). /usr/include include For Autotools install, the default value doesn't refer to prefix. But this is where general (not GCC) purpose headers should go...
docdir De dicto: directory for pkg documentation, except info files. ${datarootdir}/doc/${pkgname} Installation time
infodir infodir De dicto: directory for info files for this package. The documentation doesn't tell about localized versions. ${datarootdir}/info share/info Installation time. Curiously (or not), the documentation does not state that there should be a pkgname subdirectory.
htmldir De dicto: documentation in html format. If there are localized versions, the localized versions should be put in resp. ISO ll subdir. ${docdir} Installation time.
dvidir De dicto: documentation in (TeX) dvi format. If there are localized versions, the localized versions should be put in resp. ISO ll subdir. ${docdir} Installation time.
pdfdir De dicto: documentation in pdf® format. If there are localized versions, the localized versions should be put in resp. ISO ll subdir. ${docdir} Installation time.
psdir De dicto: documentation in PostScript® format. If there are localized versions, the localized versions should be put in resp. ISO ll subdir. ${docdir} Installation time.
libdir libdir Directory for object files and libraries of object code, but not executables. ${exec_prefix}/lib (Varying definitions depending on compilation system; may be inaccurate for cross-building: in this case, one needs to set it correctly) Installation time.
lispdir Directory for Emacs Lisp files. ${datarootdir}/emacs/site-lisp Installation time.
localedir localdir Directory for locale-specific message catalogs for this pkg. ${datarootdir}/locale share/locale The localized versions are further organized with an ISO ll subdirectory and a pkg subdirectory.
mandir mandir Top-level directory for man pages. ${datarootdir}/man share/man Installation time. Considered a secondary citizen for GNU.
manext File name extension for the installed man page, including normally a leading dot `.' (generally: `.1'). Installation time.
man[1-9]ext File name extension for the installed some num section man page. Installation time. Exist to allow installation of several different sections, instead of the uniq manext.
srcdir Directory for the sources being compiled. Compile time.
licensedir Licenses directory (since meson 1.1.0) (empty by default; used only if set; see meson.install_dependency_manifest()) Installation time.

Trimming the pkg-config file

The pkg-config file is generated from a template, having the .in extension as is traditional with Autotools for files to be processed by configure (despite being generated via a target in the Makefile, eventually configure—in fact, config.status—is called to interpolate replacement values).

The variables in this pc file are consumed solely by pkg-config are are meant to simplify and to make clear the relationships between, for example, directories, for example. The names of the variables are not to be considered related to the GNU standard. They are generally corresponding only by side-effect, because the definitions are replaced using Autotools default facility for pkg-config files. Only the @some_var@ magic strings are replaced.

The Meson variables names can be diverging, the only purpose being to provide, as said before, at least the same configuration possibilities as the Autotools framework.

There is, in every case, something that can and has to be done to simplify the understanding of what needs to be done: suppress from the pc file template whatever variable definition that is simply not used. And to retain only the informations about this very module.

In the existing template, apparently prefix, exec_prefix, datarootdir and datadir are useless. But this is not the case because of default values given to the AC_SUBST variable xcbincludedir, that is defined this way in configure.ac: xcbincludedir='${datadir}/xcb'. Hence the default value will need datadir to be defined; but the default value for datadir, with Autotools, uses datarootdir—see the table above and the commentaries. So we need to retain both for Autotools. What about the prefix and the exec_prefix? The former is used in the default definition of datarootdir as well as PYTHON_PREFIX, and we will use it to for Meson relative paths too, so needs to be retain. But exec_prefix is not used either by the default definitions of xcbincludedir or pythondir. So we suppress it:

xcb-proto.pc.in

prefix=@prefix@
exec_prefix=@exec_prefix@
datarootdir=@datarootdir@
datadir=@datadir@
xcbincludedir=${pc_sysrootdir}@xcbincludedir@
PYTHON_PREFIX=@PYTHON_PREFIX@
pythondir=${pc_sysrootdir}@pythondir@

Name: XCB Proto
Description: X protocol descriptions for XCB
Version: @PACKAGE_VERSION@

Adding to meson missing switches and variables

To list all the settings, one can invoke autoconf -t AC_SUBST. Invoking configure –help will list all the configuration options proposed. The default values for the default variables can be retrieved from the table above; what is of more interest is the added configuration options, listed in the Optional Features: or Optional Packages:, as well the Some influential environment variables:.

But the problem is that not all the options displayed are used for this package. In fact a bunch of options are default ones and are no use for our example module. So one can hardly skip looking precisely at configure.ac and Makefile.am and, in this latter file, by looking at SUBDIRS to inspect what is really used, in what way, and to then set what has to be added.

xcbincludedir is set as ${datadir}/xcb in configure.ac and is not configurable. We define it the same way, without adding on option to set it differently.

In our example, the following has to be added:

For the default definitions of the variables, that have to be interpolated in the pkgconf file, we will use the ${foo}/${bar} form in order to keep the relationship, inside the pkgconf file, between directories and to preserve the ability to change all by only changing one value (say prefix) when moving the file around. But this will not be used by Meson itself, for other purposes: for Meson own processing, the values will be expanded (the chunk of code to do so is described below).

meson_options.txt

#
# PYTHON can be set in the environment for the find_vpython(1) utility.
#
option('check_xml', type: 'boolean', value: false,
	description: 'check xml files against schema with xmllint')
option('compile_python', type: 'boolean', value: true,
	description: 'compile python at install time (self target)')
option('pkgconfdir', type: 'string', value: '${datadir}/pkgconfig',
	description: 'installation directory for pkg-config file')
option('python_prefix', type: 'string', value: '${prefix}',
	description: 'prefix for python module installation')

Setting variables

Since pythondir has to be set depending on the executable called, we need to first verify for python and define this variable accordingly. This will be done in the xcbgen/meson.build, and the root meson.build will inherit the definitions. The version that may have some ${prefix} definition will be called pc_pythondir while for Meson we will use a pythondir with these variables expanded.

As already written in the root meson.build, the srcgen/meson.build subdir will be processed first so that we will have, in the root meson.build the variables pc_python_prefix and pythondir defined.

As well, in src/meson.build will be defined a pc_xcbincludedir that will be used as is in the pkgconfig file.

So in the root meson.build we now have simply to process the pkg-config file, preserving, if possible the related hierarchy of dir definitions i.e. preserving the ${some_var} writings.

This is done for consistency and for ergonomy, allowing to visualize the relationships between the directories and allowing to move the file by changing just the definition of the prefix, without having to rewrite every definition.

But for testing purposes in the C.I., the comparison between the two generated pkg-config files will be done by comparing them semantically i.e. by comparing them with variables definitions expanded.

Meson has a predefined function to process the @SOME_VAR@ magic strings. We need to provide only the definitions that we pass as a dictionnary created by a call to configuration_data().

We set explicitely absolute paths (absolute minus DESTDIR) instead of relying on implicit pre-pending of prefix.

meson.build

pc_conf = configuration_data({
    'prefix': get_option('prefix'),
    'datarootdir': '${prefix}' / get_option('datadir'),
    'datadir': '${datarootdir}',
    'PYTHON_PREFIX': pc_python_prefix,
    'pythondir': '${PYTHON_PREFIX}' / pythondir,
    'xcbincludedir': pc_xcbincludedir,
    'PACKAGE_VERSION': meson.project_version()
})

pkgconfdir = get_option('pkgconfdir')
pkgconfdir = pkgconfdir.replace('${datadir}', datadir)
configure_file(input: 'xcb-proto.pc.in',
               output: 'xcb-proto.pc',
               install_dir: get_option('prefix') / pkgconfdir,
               configuration: pc_conf)

Testing for xmllint

We handle the xml files in the src/ subdirectory, that has to have a meson.build file too, since it is called by the root meson.build via the subdir('src') instruction.

We have added an option to require explicitely checking of the xml files with xmllint. Since we require explicit instruction to check, failure, whether to find a xmllint or to check shall be fatal.

A function to search for a program is given. A function to execute an external command is given too and we can require that a failure is fatal. So this is a basic usage of the facilities provided.

We start by defining an array of the files to handle, since they also will have to be installed. A function is also provided for this.

src/meson.build

prefix = get_option('prefix')
datadir = get_option('datadir')

#!!! This is for pkgconf file where '${datadir}' is "prefixed"
pc_xcbincludedir = '${datadir}/xcb' # this one for pkgconf file

xcbincludedir = prefix / datadir / 'xcb'

xml_files = [
    'xcb.xsd',
    'xproto.xml',
    'bigreq.xml',
    'composite.xml',
    'damage.xml',
    'dbe.xml',
    'dpms.xml',
    'dri2.xml',
    'dri3.xml',
    'ge.xml',
    'glx.xml',
    'present.xml',
    'randr.xml',
    'record.xml',
    'render.xml',
    'res.xml',
    'screensaver.xml',
    'shape.xml',
    'shm.xml',
    'sync.xml',
    'xc_misc.xml',
    'xevie.xml',
    'xf86dri.xml',
    'xf86vidmode.xml',
    'xfixes.xml',
    'xinerama.xml',
    'xinput.xml',
    'xkb.xml',
    'xprint.xml',
    'xselinux.xml',
    'xtest.xml',
    'xv.xml',
    'xvmc.xml'
]

# CHECKING IF REQUESTED
# If checking is requested, failure is fatal.
#
if get_option('check_xml')
    xmllint = find_program('xmllint', required : true)
    foreach f : xml_files
        if f != 'xcb.xsd'
            run_command(xmllint, '–noout', '–schema', 'xcb.xsd',
               f, check: true)
        endif
    endforeach
endif

install_data(xml_files, install_dir: xcbincludedir)

Handling the python files and problems

The main problem for our example module is Python.

It is not required for this module (we do not need to run a python interpreter). But the directory where to place python third parties packages depends on the system et on the python version...

Furthermore, it is customary to byte-compile the python package in order both to speed-up the execution (Python is interpreted) and to install the byte-compiled version in a hierarchy that perhaps not every user can write—and not having to byte-compile on every call, if the program is configured to do so, because there is already such a version in one known place, is a gain of time too.

So we will try to find a python interpreter matching the version of what has been written, but failing to do so is not fatal, and if requested (this is requested by default since we have added an option allowing to tell to not byte compile by default) we will byte-compiled.

In the cross-compilation case, finding an interpreter may succeed but without giving an authoritative answer (this interpreter is not the version of what will be found in the target for the installation), and byte-compilation will not work since it is not arch or system independent.

The version needed for the python module is found in configure.ac: AM_PATH_PYTHON([2.5]).

So here, we will be using a POSIX.2 shell script to search for a python interpreter matching the required version (>= 2.5), named find_vpython.sh, that is not part of Meson installation and that has been written specifically for the task, and that returns on stdout two informations: the pathname of the matching interpreter, and the definition of the pythondir (minus a python_prefix that has to be defined).
If the program is found (we have defined a boolean has_python to keep the definition), and if this is requested, after installation, the Python package will be compiled, using another ad hoc POSIX.2 shell script written: compile_py.sh.

Both utilities are placed at the root of the module, hence the relative prefix when calling.

xcbgen/meson.build

# The pythondir depends on Python version needed. We call an external
# utility for this but for this module the absence of python is not
# fatal: all in all, the *.py files are not used _here_ and python
# is only needed to pre-compile at installation time.
# It returns on stdout
#
python_search = run_command('../find_vpython.sh', '>=', '2', '5', check : false)
has_python = false
if python_search.returncode() == 0
    has_python = true
    python = python_search.stdout().split()[0]
    pythondir = python_search.stdout().split()[1]
    message('python found:', python)
else
    message('python not found:')
    # Agnostic definition of the subdir where to install
    pythondir = 'lib/python/site-packages'
endif

pc_python_prefix = get_option('python_prefix')
python_prefix = pc_python_prefix.replace('${prefix}', get_option('prefix'))

py_files = [
    '__init__.py',
    'align.py',
    'error.py',
    'expr.py',
    'matcher.py',
    'state.py',
    'xtypes.py'
]

install_data(py_files, install_dir: python_prefix / pythondir / 'xcbgen')
if has_python and get_option('compile_python')
    meson.add_install_script('../compile_py.sh', python,
        python_prefix / pythondir / 'xcbgen', '1')
endif

What has to be added to Autotools files

When the module is distributed, all the files, including the Meson ones, have to be distributed.

The tarball can be created using the Autotools framework. Hence we must instruct it to include the Meson files added.

This is done in the root Makefile.am:

Makefile.am

EXTRA_DIST += meson.build meson_options.txt
EXTRA_DIST += src/meson.build xcbgen/meson.build
EXTRA_DIST += find_vpython.sh compile_py.sh

Meson dist command packs the whole committed src tree and not a specifically defined list of files.

Autoconf idiosynchrasies

AC_CONFIG_SRCDIR is uniq to autoconf, giving a filename that is uniq to the package to be configured in order to verify the srcdir by checking its presence. In this case, the pkg-config template matches the criterium, but has nothing to do with the actual generation of the pc file.

The C.I. facility

References

The Continuous Integration in short

The principle of the Continuous Integration (C.I.) is that once a developer has validated changes in a fork, he requests a merge on the master which, once accepted, triggers a check of the changes by automated building of the module and tests of the results.

In the git tree, the file driving the building, checking and so on is .gitlab-ci.yml. Once the modifications of the module have been done, this file has to be adapted and committed before requesting the merge.

The file for each module—as it should be—commented, so we will only expose below some points about it.

In a nutshell, after including pre-defined instructions, specially the kind of VM (container) to use, the stages defines the ordered list of stages (group of jobs; in a group, jobs are independent and can run concurrently) to execute, and then come the definitions of variables and the definition (instructions) of each job.

The stages are sequential; but a stage is a group of jobs, and, in a group (stage), jobs are independent, running (if possible) concurrently. One can pass things between stages, not between jobs inside a group (stage).

In the instructions, one will find pushd and popd. For strictly POSIX.2 shell users (*BSD programmers for example), it shall be noted that these are not special commands of the C.I. language, but bash'ismes: these add or remove from a stack of directories, allowing, in the general case, to return to a directory without caring about what the current directory is. This has to be taken into account if some *BSD container was to be added: bash is generally expected.

Augmenting the number of stages?

For clarity, we will add a stage to specifically check for discrepancies between Autotools and Meson. But, all in all, we could probably have simply added steps to the test stage.

The stage added will be called check.

stages:
  - prep             # prep work like rebuilding the container images if there is a change
  - build            # for actually building things in a container
  - check            # checking that autotools and meson yield same res
  - test
  - deploy

When comparing the meson and the autotools results, we will need an utility to compare the generated pkg-config files but, as explained above, we will do it with an utility that compares the two files semantically, allowing to not use the same variables names but requiring that a default configuration yields, eventually, the same definitions. This tool is a POSIX.2 shell script called auto-meson-pc-cmp.sh living in xorg/util/modular. We will simply clone the git repository in the step where the comparison is made.

Needed tools in container (packages)

Building and testing is done in a container, for a certain OS. It has to provide all the tools needed to build and test the module.

Since it uses the packaging facilities for the OS, adding a tool is specifying the name of the package that contains it. For example xmllint is an utility provided by libxml2 and this is this latter name that one will find in the definition of the packages to add.

Furthermore, since something has to be changed in the VM, we need to instruct to rebuild the container. This is done via the definition of FDO_DISTRIBUTION_TAG, that has to be changed in order to trigger the rebuilt.

Obviously for our purpose, meson and ninja tools have to be added. They are, for example on Arch Linux—the container requested by file: '/templates/arch.yml' in our example case—provided by the homonymous packages:

In order to be able to compare the byte-compiled version of the Python modules, we need to ensure that these versions are in theory identical. For this, we install the same hierarchy under two distinct subdirs using DESTDIR, and the utilities byte-compiling remove the DESTDIR prefix from the paths embedded in the byte-compiled version.

But there is also a date embedded in the byte-compiled version. Fortunately, Python supports SOURCE_DATE_EPOCH. If it is set in the environment, it is used as the date. So we set it as a global variable of the pipeline.

 FDO_DISTRIBUTION_TAG:  '2022-07-23.0'
 FDO_DISTRIBUTION_TAG:  '2024-06-14.0'
   # Packages needed to build xcbproto
 XCBPROTO_PACKAGES:  'git gcc pkgconf autoconf automake make libxml2 python'
 XCBPROTO_PACKAGES:  'git gcc pkgconf autoconf automake make meson ninja libxml2 python'
  # Additional packages needed to build libxcb
  LIBXCB_PACKAGES:  'libtool xorg-util-macros doxygen graphviz check libxslt libxau libxdmcp'
  FDO_DISTRIBUTION_PACKAGES: $XCBPROTO_PACKAGES $LIBXCB_PACKAGES
  # Forcing to not use the time of compilation for Python (check)
  SOURCE_DATE_EPOCH: $(date "+%FT%T")

The name XCBPROTO_PACKAGES is not a mandatory name formed of module_name with a _PACKAGES suffix. It is just a variable name, that is used further in code in the definition of the reserved FDO_DISTRIBUTION_PACKAGES: FDO_DISTRIBUTION_PACKAGES: $XCBPROTO_PACKAGES $LIBXCB_PACKAGES.

Adding steps

In fact we will be almost only adding steps to the previous version, leaving the previous names (almost) alone.

These steps were building a version with Autotools and then verifying that libxcb could be built with the result.

The only modification is that we preserve a tarball created by Autotools in order to use it to verify, later, that Meson has all the required files in this tarball to be able to do the job.

The Autotools build step was called build (homonymous to the stage). We let it alone.

build:
  stage: build
  extends:
    - .fdo.distribution-image@arch
  script:
    - export INSTDIR="$PWD/_inst"
    - mkdir _builddir
    - pushd _builddir > /dev/null
    - ../autogen.sh –disable-silent-rules –prefix="$INSTDIR"
    - make
    - make check
    - make install
    - make distcheck	# creates the tarball
    - mv xcb-proto-*.tar.xz ..
    - popd > /dev/null
  variables:
  artifacts:
    paths:
      - _inst
      - xcb-proto-*.tar.xz

Adding another building with Autotools

For checking Autotools against Meson, as said above, we need to generate the very same hierarchy—so we set DESTDIR—and the very same date for Python byte-compiled files—so we set SOURCE_DATE_EPOCH from the global definition we have added to the pipeline.

And we need to preserve the result, so we declare an artifact.

autotools:
  stage: build
  extends:
    - .fdo.distribution-image@arch
  script:
    - export SOURCE_DATE_EPOCH=$SOURCE_DATE_EPOCH
    - export DESTDIR="$PWD/_autotools_inst"
    - mkdir _builddir0
    - pushd _builddir0 > /dev/null
    - ../autogen.sh –disable-silent-rules
    - make install
    - popd > /dev/null
  variables:
  artifacts:
    paths:
      - _autotools_inst

Adding building with Meson

We add a step for Meson building and installation and, here also, we keep the installed result for a further test (artifacts): we want to ensure that the results are, for the same configuration, the same whether with Autotools or with Meson.

In order to have the same result, we need to pass a supplementary option to Meson: autoconf, by default, checks XML syntax if xmllint is available. It is, because the package libxml2 is added. So we explicitely instruct Meson to do the same—because we have added an option to ask explicitely for this, that is mainly of use for xcb developers themselves, and not end user.

What has been said about Autotools applies here: we set DESTDIR and SOURCE_DATE_EPOCH.

And we keep the result declaring an artifact as well.

meson:
  extends:
    - .fdo.distribution-image@arch
  stage: build
  script:
    - export SOURCE_DATE_EPOCH=$SOURCE_DATE_EPOCH
    - export DESTDIR=$PWD/_meson_inst
    - meson setup builddir -Dcheck_xml=true
    - meson install -C builddir
  artifacts:
    paths:
      - _meson_inst

Testing Autotools and Meson same result

We have kept the installed result in artifacts paths _autotools_inst and _meson_inst. So we will compare that the same installed result is achieved—because we are compiling on a same self-host, and we have passed the same configuration—with one exception: the pkg-config files could have differing names in the definition of variables, in order to not hamper Meson configuration with Autotools idiosynchrasies but the results (definitions expanded) shall be the same. So we need to compare the pkg-config generated files semantically and not literally.

There is a POSIX.2 shell utility in xorg/util/modular called auto-meson-pc-cmp.sh for doing so, so we simply clone the git repository (nothing to compile or install) to call it, and we exclude the pc files from the further comparison.

Since the version is set in the pkg-config file, we do not test separately for version consistency.

#
# The configuration for Autotools and Meson generation/installation
# shall be so to yield the same result.
# Since the version is in the pc file, we do not test for version
# identity separately but we do compare pc generated files
# separately in order to compare them semantically and not literaly.
#
compare meson and autotools:
  extends:
    - .fdo.distribution-image@arch
  stage: test
  script:
    - git clone –depth=1 https://gitlab.freedesktop.org/xorg/util/modular
    - modular/auto-meson-pc-cmp.sh $PWD/_meson_inst $PWD/_autotools_inst
    - diff –brief –recursive -x '*.pc' $PWD/_meson_inst $PWD/_autotools_inst

Verifying that Meson can work with an Autotools created tarball

We have created (and preserved as artifact) a distribution tarball with Autotools. The aim, here, is to verify that the Meson required files have indeed been added to the distribution files, so that Meson will work with an Autotools generated tarball.

Should be self-explanatory.

meson from autotools tarball:
  extends:
    - .fdo.distribution-image@arch
  stage: check
  script:
    - mkdir -p _tarball_build
    - tar xf xcb-proto-*.tar.xz -C _tarball_build
    - pushd _tarball_build/xcb-proto-*
    - meson setup builddir -Dcheck_xml=true
    - meson install -C builddir
  variables:
    GIT_STRATEGY: none

©2022–2024 Thierry Laronde — FRANCE