#+TITLE:  Build Generator
#+AUTHOR: Jan Moringen
#+EMAIL:  jmoringe@techfak.uni-bielefeld.de

#+LINK: jenkins   https://jenkins.io/
#+LINK: sbcl      http://sbcl.org
#+LINK: quicklisp https://www.quicklisp.org/beta/

#+ATTR_HTML: :alt "build status image" :title Build Status :align right
[[https://travis-ci.org/RDTK/generator][https://travis-ci.org/RDTK/generator.svg]]

* Introduction

  The build generator is a program for building the components of software systems from declarative recipes describing software projects and software systems (and a few other things).
  This can be done in several ways.
  One way is automatically installing and configuring a [[jenkins][Jenkins]] instance, then automatically generating Jenkins jobs which do the actual building.
  Other ways include generating [[https://docs.docker.com/engine/reference/builder/][Dockerfiles]], generating [[https://www.gnu.org/software/make/manual/make.html#Introduction][Makefiles]] and building in the local filesystem under the control of the build generator process.

* Obtaining a Build Generator Binary

  One way to obtain a build generator binary is [[*Building][building from source]].
  Furthermore, the following pre-built binaries are available:

  + Binaries for released versions can be downloaded in the [[https://github.com/rdtk/generator/releases]["Releases" section of the GitHub project]].

  + Binaries for the master version can be downloaded from the [[https://jenkins-cse.bob.ci.cit-ec.net/view/build-generator-nightly-ci-docker/job/build-generator-master-ci-docker-build-generator-nightly][continous integration server]].

* Dependencies

  + Common Lisp Implementation :: At the moment, the code is almost but not quite portable and can only be used with the [[sbcl][SBCL]] implementation of Common Lisp.

  + System Libraries :: For secure communication with the Jenkins server as well as archive retrieval via HTTPS, the build generator relies on the OpenSSL library, specifically =libssl.so.*=.
       In Debian-derived systems, the =openssl= package provides this library.

  + Lisp dependencies :: The following Lisp dependencies cannot be installed automatically via [[quicklisp][Quicklisp]], yet:

       + The [[https://github.com/scymtym/model.transform.trace][model.transform.trace]] project

       + The [[https://github.com/scymtym/language.yaml][language.yaml]] project

       + The [[https://github.com/scymtym/text.source-location][text.source-location]] project

       + The [[https://code.cor-lab.org/projects/rosetta][rosetta]] project

         + The [[https://code.cor-lab.org/projects/rosetta][rosetta-project]] project

* Building

  The following steps are intended to achieve the following:

  + building the master version of the build generator

  + without elevated privileges

  + without polluting system or user directories outside of a =build= sub-directory within the project directory

  The last two points make things a little more complicated but hopefully more useful overall.

  1. Start in the project directory

     #+BEGIN_SRC bash
       git clone https://github.com/RDTK/generator
       cd generator
       mkdir -p build
     #+END_SRC

  2. Installing SBCL

     Not all Debian-like systems provide a reasonably recent version of the [[sbcl][SBCL]] implementation of Common Lisp.
     It is therefore better to install a binary release from the project website:

     #+BEGIN_SRC bash
       (
         cd build
         curl -L "https://prdownloads.sourceforge.net/sbcl/sbcl-1.5.0-x86-64-linux-binary.tar.bz2" | tar -xj
         cd sbcl-*
         INSTALL_ROOT="$(pwd)/../sbcl" sh install.sh
       )
     #+END_SRC

  3. Installing Quicklisp

     [[quicklisp][Quicklisp]] is a package manager and installer for Common Lisp.
     See [[quicklisp]] for detailed instructions including cryptographic verification of the downloaded file.

     A quick way to install Quicklisp is:

     #+BEGIN_SRC bash
       (
         cd build
         curl -o quicklisp.lisp "https://beta.quicklisp.org/quicklisp.lisp"
         SBCL_HOME="$(pwd)/sbcl/lib/sbcl"                               \
           sbcl/bin/sbcl --noinform --non-interactive                   \
             --load quicklisp.lisp                                      \
             --eval '(quicklisp-quickstart:install :path "quicklisp")'
       )
     #+END_SRC

     This creates the directory =build/quicklisp=.
     Dependencies will be downloaded into sub-directories of this directory.

  4. Installing non-Quicklisp dependencies

     A quick way to install the [[Dependencies][non-Quicklisp dependencies]] is

     #+BEGIN_SRC bash
       (
         cd build/quicklisp/local-projects                                         \
           && git clone -b future https://github.com/scymtym/model.transform.trace \
           && git clone -b future https://github.com/scymtym/language.yaml         \
           && git clone -b future https://github.com/scymtym/text.source-location  \
           && git clone https://code.cor-lab.de/git/rosetta.git.cl                 \
           && git clone https://code.cor-lab.de/git/rosetta.git.project-cl
       )
     #+END_SRC

  5. Building the executable

     #+BEGIN_SRC bash
       SBCL_HOME="$(pwd)/build/sbcl/lib/sbcl"                             \
       ASDF_OUTPUT_TRANSLATIONS="/:$(pwd)/build/fasl-cache/"              \
       CL_SOURCE_REGISTRY="$(pwd)//:"                                     \
         build/sbcl/bin/sbcl --noinform --non-interactive                 \
           --load build/quicklisp/setup.lisp                              \
           --eval '(ql:quickload :build-generator.commandline-interface)' \
           --eval '(asdf:make :build-generator.commandline-interface)'
     #+END_SRC

  This should produce a binary named =build-generator= in the project root directory.

* Running Tests

  Assuming the steps in [[Building]] have been completed, the following runs the unit tests of the project

  #+BEGIN_SRC bash
    SBCL_HOME="$(pwd)/build/sbcl/lib/sbcl"                \
    ASDF_OUTPUT_TRANSLATIONS="/:$(pwd)/build/fasl-cache/" \
    CL_SOURCE_REGISTRY="$(pwd)//:"                        \
      build/sbcl/bin/sbcl --noinform --non-interactive    \
        --load build/quicklisp/setup.lisp                 \
        --eval '(ql:quickload :build-generator/test)'     \
        --eval '(asdf:test-system :build-generator)'
  #+END_SRC

* Contributing

  If you want to contribute to this project, please

  + Submit your intended changes as coherent pull requests.

  + Rebase onto the master branch and squash any fixups and corrections.

  + Make sure the unit tests pass.

* Acknowledgments

  The development of this software has been supported as follows:

  + The development of this software was supported by CoR-Lab, Research Institute for Cognition and Robotics Bielefeld University.

  + This work was supported by the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG).