Porting Cabal packages to FreeBSD - Part 2

Back in part 1, I tried to give a basic overview of FreeBSD porting. In this part, I'll show you how I ported hs-HTTP. Actually another maintainer (Jacula Modyun) beat me to it! I think he's the most active Haskell maintainer. Anyway, I'm going to walk you through all steps required to port the Cabal package to a FreeBSD port. Let's start with the sources. The source tarball of the HTTP package is available on Hackage.

Next install the tools:

#cd /usr/ports/ports-mgmt/portlint && make install clean

#cd /usr/ports/ports-mgmt/porttools && make install clean

#cd /usr/ports/ports-mgmt/genplist && make install clean

This will install a binary called port (part of the porttools). When you call it the first time it generates the configuration file .porttools in your home directory. Try it.

#port

===>; Generating /root/.porttools configuration file

FreeBSD Port Tools 0.99

Usage:    port []

Available commands:
commit  - commit a port into the FreeBSD Ports CVS Repository
create  - create new port from template using newfile(1)
diff    - generate diff against original port version
fetch   - fetch distfile(s) of new port version
getpr   - get patch/shar from a PR
help    - display this command summary
install - install a port
submit  - submit Problem Report with new port, or port change/update
test    - test port (build, install, package, deinstall)
upgrade - upgrade a port

You should now edit the newly created .porttools file. It contains settings that will be used to create template files for your port. You should at least put in your correct email address. What settings are supported can be found in the manpage (man 5 porttools).

# FreeBSD Port Tools configuration file - see porttools(5)

# vim: ft=sh

EMAIL="joeyjojo@mail.com"

FULLNAME="Joeyjojo Shabadoo"

ORGANIZATION=""

BUILDROOT="/tmp"

ARCHIVE_DIR=""

DIFF_MODE="CVS"

DIFF_VIEWER="more"

PORTLINT_FLAGS="abct"

Creating the templates

I prefer to create new ports in /tmp. To create a new directory for the port HTTP do the following:

#cd /tmp

#port create HTTP

porttools will also create some template files for us:

#ls /tmp/HTTP

Makefile    pkg-descr    pkg-plist

We can now go and set the variables CATEGORIES, COMMENT, PORTVERSION, MASTER_SITES and PKGNAMEPREFIX.

#cd /tmp/hs-HTTP

#cat Makefile

# Whom:                 Joeyjojo Shabadoo
#
# $FreeBSD$
#

PORTNAME=       HTTP
PORTVERSION=    4000.0.9
#PORTREVISION=  0
#PORTEPOCH=     0
CATEGORIES=     www haskell
MASTER_SITES=   http://hackage.haskell.org/packages/archive/${PORTNAME}/${PORTVERSION}/
#MASTER_SITE_SUBDIR=
PKGNAMEPREFIX=  hs-
#PKGNAMESUFFIX=
#DISTNAME=
#EXTRACT_SUFX=
#DISTFILES=
#DIST_SUBDIR=   ${PORTNAME}
#EXTRACT_ONLY=

MAINTAINER=     joeyjojo@mail.com
COMMENT=        A library for client-side HTTP

.include <bsd.port.pre.mk>

.include <bsd.port.post.mk>

The PORTVERSION and the COMMENT were directly copied from the Hackage website. I chose CATEGORIES www and haskell. Note that the category haskell is only virtual and can therefore never be declared at the first position. www on the other hand is a real category which fits very nicely for our HTTP-client library. Refer the Porter's Handbook for more information on proper categorization and a list of existing categories.

The PKGNAMEPREFIX for all Haskell ports is 'hs-'. So you should always set that as well.

And finally MASTER_SITES contains a list of URLs where the source tarball is located. It is recommended to specify a list rather than a single location for redundancy reasons. Unfortunately I know only the one URL specified on Hackage.

A word about formatting

There are a couple of formatting rules you have to follow. If you don't, portlint will tell you about it.

• always use tabs instead of spaces for indentation. Also, indent all variable assignments.

• you mustn't put consecutive empty lines into the Makefile. 

• avoid spaces or tabs at the end of a line 

First test run - fetching the source

At this point the Makefile has enough information to download the tarball and calculate the checksums. So lets try that to verify that the URL is correct.

#port fetch
=> HTTP-4000.0.9.tar.gz doesn't seem to exist in /usr/ports/distfiles/.

=> Attempting to fetch from http://hackage.haskell.org/packages/archive/HTTP/4000.0.9/.

HTTP-4000.0.9.tar.gz                          100% of   58 kB   40 kBps

#cat distinfo

MD5 (HTTP-4000.0.9.tar.gz) = bbd005935537ed8883bfefb624e8bf3c

SHA256 (HTTP-4000.0.9.tar.gz) = 1e2b4a8b782ad1417c8755bb0d248851bc142b351366ed460e07f2945a5e95ba

SIZE (HTTP-4000.0.9.tar.gz) = 59528

Documentation

So far I haven't talked about how to install any documentation. The variable NOPORTDOCS controls whether or not documentation should be installed. Most Haskell modules contain documented source files. This documentation can be extracted and converted to browsable HTML files using Haddock.

Haddock

Haddock can be either built into ghc or be installed as separate package. This can be checked through make -V PORT_HADDOCK. It will return a two digit number indicating whether or not haddock was installed as separate port.

PORT_HADDOCK=  (cd  ${PORTSDIR}/lang/ghc && ${MAKE} -V PORT_HADDOCK)

You can try it on the shell prompt to see what it does:

#cd /usr/ports/lang/ghc && make -V PORT_HADDOCK

#10

If Haddock hasn't been integrated into ghc, we have to declare it as additional build dependency so that it will be installed if necessary. PORT_HADDOCK:M?0 will match either "10" or "00". In case you wonder, the meaning of these variable values is defined inside ghc's Makefile (/usr/ports/lang/ghc) file. It is much more involved than the average Haskell port but still worth a look.

.if !empty(${PORT_HADDOCK:M?0})

BUILD_DEPENDS+= haddock:${PORTSDIR}/devel/hs-haddock

.endif

BTW, all options, commands, conditionals, variable assignments and pattern matching used in the Makefile are documented in make's manpage (man 1 make).

In addition to extracting documentation, Haddock can also convert, format and highlight the source code itself using the port hs-hscolour. So that means another build dependency.

BUILD_DEPENDS+= HsColour:${PORTSDIR}/print/hs-hscolour

Finally, we also have to set the variable PORTDOCS. It is intended to hold a list of all conditionally installed documentation. Which will also be included into the final packing list. The Handbook specifies:

If a directory is listed in PORTDOCS or matched by a glob pattern from this variable, the entire subtree of contained files and directories will be registered in the final packing list. If NOPORTDOCS is defined then files and directories listed in PORTDOCS would not be installed and neither would be added to port packing list.

To give you an idea, the whole block looks like this:

.ifndef(NOPORTDOCS)

CHECK_HADDOCK=  (cd  ${PORTSDIR}/lang/ghc && ${MAKE} -V PORT_HADDOCK)

.if !empty(${CHECK_HADDOCK:M?0})

BUILD_DEPENDS+= haddock:${PORTSDIR}/devel/hs-haddock

.endif

BUILD_DEPENDS+= HsColour:${PORTSDIR}/print/hs-hscolour

HSCOLOUR_VERSION=       1.15

HSCOLOUR_DATADIR=       ${PREFIX}/share/hscolour-${HSCOLOUR_VERSION}

PORTDOCS=       *

.endif

The block is identical for virtually every Haskell port. But I think it is still important to understand what all this stuff means.

Other Documentation

There some more variables, that influence pkg-plist, which are related to PORTDOCS that you should know about.

DOCSDIR: by default this variable will get expanded to PREFIX/share/doc/PORTNAME. But we'll redefine it to DOCSDIR=${PREFIX}/share/doc/${DISTNAME}. Where DISTNAME is just PORTNAME plus PORTVERSION. This way we can install different docs for different versions of the same port.

PLIST_SUB: If you need to make other substitutions, you can set this variable with a list of VAR=VALUE pairs and instances of %%VAR%% will be substituted with VALUE in the pkg-plist.

If your port installs files conditionally on the options set in the port, the usual way of handling it is prefixing the pkg-plist lines with a %%TAG%% and adding that TAG to the PLIST_SUB variable inside the Makefile with a special value of @comment, which makes package tools ignore the line. So in our case we do:

.ifdef(NOPORTDOCS)

PLIST_SUB+=     NOPORTDOCS=""

.else

PLIST_SUB+=     NOPORTDOCS="@comment "

.endif

If the port was installed without additional documentation, it will still put a LICENSE file in its doc folder - /usr/local/share/doc/HTTP-4000.0.9 in our case. So that will be accounted for in the pkg-plist.

Defining the Make targets

In order to configure, build and install the port, we have to define the corresponding targets. I will also define the two variables SETUP_CMD = ./setup and GHC_CMD = ${LOCALBASE}/bin/ghc (LOCALBASE normally points to /usr/local/).

Which reminds me. I need ghc to be present on the system to build as well as to run the HTTP library. So I have to declare BUILD_DEPENDS and RUN_DEPENDS. If it isn't, it will be automatically installed as a dependency.

BUILD_DEPENDS+= ghc:${PORTSDIR}/lang/ghc

RUN_DEPENDS+=   ghc:${PORTSDIR}/lang/ghc

We can then copy the default targets which I've also shown in the first part. They too are identical for most ports. The only important thing you might have to adjust is the Setup source. Some packages contain Setup.lhs while others use Setup.hs, you have to pick the correct one in the do-configure target.

do-configure:

        cd ${WRKSRC} && ${GHC_CMD} --make Setup.lhs \
                                   -o setup -package Cabal \

                     && ${SETUP_CMD} configure \
                      --haddock-options=-w --prefix=${PREFIX}

do-build:

        cd ${WRKSRC} && ${SETUP_CMD} build \

                     && ${SETUP_CMD} register --gen-script

.if !defined(NOPORTDOCS)

        cd ${WRKSRC} && ${SETUP_CMD} haddock \

                 --hyperlink-source \

                 --hscolour-css=${HSCOLOUR_DATADIR}/hscolour.css

.endif

do-install:

        cd ${WRKSRC} && ${SETUP_CMD} install \
                     && ${INSTALL_SCRIPT} register.sh ${PREFIX}/${HTTP_LIBDIR_REL}/register.sh

post-install:

        ${RM} -f ${PREFIX}/lib/ghc-${GHC_VERSION}/package.conf.old

Note: The command setup register --gen-script in the do-build target registers this package with the compiler, i.e. makes the modules it contains available to programs. The setup install command actually incorporates this action. The main use of this separate command is in the post-installation step for a binary package.

The whole Makefile now looks like this:

# New ports collection makefile for:    HTTP
# Date created:         2010-01-28
# Whom:                 your name
#
# $FreeBSD$
#

PORTNAME=       HTTP
PORTVERSION=    4000.0.9
CATEGORIES=     www haskell
MASTER_SITES=   http://hackage.haskell.org/packages/archive/${PORTNAME}/${PORTVERSION}/
PKGNAMEPREFIX=  hs-

MAINTAINER=     your.name@mail.com
COMMENT=        A Haskell library for client-side HTTP

BUILD_DEPENDS=  ghc:${PORTSDIR}/lang/ghc
RUN_DEPENDS+=   ghc:${PORTSDIR}/lang/ghc

GHC_VERSION=    6.10.4
HTTP_VERSION=   ${PORTVERSION}

GHC_CMD=        ${LOCALBASE}/bin/ghc
SETUP_CMD=      ./setup

DOCSDIR=                ${PREFIX}/share/doc/${DISTNAME}
HTTP_LIBDIR_REL=        lib/${DISTNAME}

PLIST_SUB=      GHC_VERSION=${GHC_VERSION} \
                HTTP_VERSION=${HTTP_VERSION} \
                HTTP_LIBDIR_REL=${HTTP_LIBDIR_REL}

.ifdef(NOPORTDOCS)
PLIST_SUB+=     NOPORTDOCS=""
.else
PLIST_SUB+=     NOPORTDOCS="@comment "
.endif

.ifndef(NOPORTDOCS)

PORT_HADDOCK=   (cd  ${PORTSDIR}/lang/ghc && ${MAKE} -V PORT_HADDOCK)
.if !empty(PORT_HADDOCK:M?0)
BUILD_DEPENDS+= haddock:${PORTSDIR}/devel/hs-haddock
.endif
BUILD_DEPENDS+= HsColour:${PORTSDIR}/print/hs-hscolour

HSCOLOUR_VERSION=       1.15
HSCOLOUR_DATADIR=       /usr/local/share/hscolour-${HSCOLOUR_VERSION}

PORTDOCS=       *
.endif

.SILENT:

do-configure:

        cd ${WRKSRC} && ${GHC_CMD} --make Setup.lhs \
                                   -o setup -package Cabal \

                     && ${SETUP_CMD} configure \
                      --haddock-options=-w --prefix=${PREFIX}

do-build:

        cd ${WRKSRC} && ${SETUP_CMD} build \

                     && ${SETUP_CMD} register --gen-script

.if !defined(NOPORTDOCS)

        cd ${WRKSRC} && ${SETUP_CMD} haddock \

                 --hyperlink-source \

                 --hscolour-css=${HSCOLOUR_DATADIR}/hscolour.css

.endif

do-install:

        cd ${WRKSRC} && ${SETUP_CMD} install \
                     && ${INSTALL_SCRIPT} register.sh ${PREFIX}/${HTTP_LIBDIR_REL}/register.sh

post-install:

        ${RM} -f ${PREFIX}/lib/ghc-${GHC_VERSION}/package.conf.old

Port Description

We need to put a description into pkg-descr. I usually just copy whatever description is already part of the source. So that's straight forward.

Packing List

The last missing file is pkg-plist. It contains the listing of all files and directories created by our port. The packing list is very important, without it ports could not be cleanly deinstalled.

Handling the documentation

As we've seen in the paragraph about Documentation, you can specify PORTDOCS and PLIST_SUB dynamically within the Makefile instead of explicitly listing every file in pkg-plist. So the whole documentation installed in /usr/local/share/doc/YourPort/ is already taken care of. That's very convenient because sometimes it is difficult to tell beforehand what files will be installed as part of the documentation.

Handling the no-documentation

Remember, I've defined a substitution similar to the one above for NOPORTDOCS in case no documentation should be installed. In that case the LICENSE file will be put into the port's doc-folder nontheless. That, we have to deal with. So I'm going to add the following two lines to pkg-plist:

%%NOPORTDOCS%%%%DOCSDIR%%/LICENSE
%%NOPORTDOCS%%@dirrmtry %%DOCSDIR%%

Ok, so what does this do? Remember what we've defined in the Makefile:

.ifdef(NOPORTDOCS)

PLIST_SUB+=     NOPORTDOCS=""

.else

PLIST_SUB+=     NOPORTDOCS="@comment "

.endif

That means if NOPORTDOCS is defined, %%NOPORTDOCS%% will be substituted with and empty string. So the final packing list will contain the two lines:

%%DOCSDIR%%/LICENSE
@dirrmtry %%DOCSDIR%%

The @dirrmtry is an instruction to remove this port's doc-folder when it is being deinstalled, but only if that directory is empty. Otherwise it won't do anything. There is also the related @dirrm instruction. It will fail if the directory to remove in not empty. There is no such thing as @filerm to remove files. All listed files will be removed automatically during deinstallation.

On the other hand, if  NOPORTDOCS is not defined, %%NOPORTDOCS%% will be replaced with "@comment " and we end up with:

@comment %%DOCSDIR%%/LICENSE
@comment @dirrmtry %%DOCSDIR%%

All lines beginning with @comment will be ignored in the final packing list. So both cases are covered.

 

Handling pre-built packages

Until now I always explained things from the vantage point of what happens when you compile the port from source. But FreeBSD also supports the installation of pre-built packages (through pkg_add). We have to consider this as well. Think about it. We can easily compile a library and put it into a package. But when installing such a package, you also have to register the library. In the case of Haskell libraries, we have to register them with ghc. This is also done in pkg-plist with another two directives, @exec and @unexec respectively:

@exec /bin/sh %D/%%HTTP_LIBDIR_REL%%/register.sh
@exec /bin/rm -f %D/lib/ghc-%%GHC_VERSION%%/package.conf.old

This will run the register-script we've built in the do-build target. That script will be part of the package.

@unexec %D/bin/ghc-pkg unregister HTTP
@unexec /bin/rm -f %D/lib/ghc-%%GHC_VERSION%%/package.conf.old

Unexec is the reverse operation that unregisters the Haskell module as part of the deinstallation process of the port. Be careful to unregister the correct one.

Handling the rest of the files

I promised to show you how to automatically generate pkg-plist. However, all of the above has to be managed manually. So what's left you ask? All the files that are compiled from the Haskell source. I'll use genplist to track those. The path /tmp/plists is the working directory of genplist. genplist's man page suggests using /tmp but that would cause a conflict since our  port is located there.

#mkdir /tmp/plists && genplist create /tmp/plists

 PORTNAME = HTTP
PREFIX = /tmp/plist/HTTP
===>  Deinstalling for www/hs-HTTP
===>   hs-HTTP not installed, skipping
===>   hs-HTTP-4000.0.9 depends on executable: ghc - found
===>   hs-HTTP-4000.0.9 depends on executable: haddock - found
===>   hs-HTTP-4000.0.9 depends on executable: HsColour - found
===>   hs-HTTP-4000.0.9 depends on executable: ghc - found
./bin missing (created)
./etc missing (created)
./etc/pam.d missing (created)
./etc/rc.d missing (created)
./include missing (created)
./include/X11 missing (created)
./info missing (created)
./lib missing (created)
./lib/X11 missing (created)
./lib/X11/app-defaults missing (created)
./lib/X11/fonts missing (created)

...

...

ar: warning: creating dist/build/libHSHTTP-4000.0.9.a
Writing registration script: register.sh for HTTP-4000.0.9...
Preprocessing library HTTP-4000.0.9...
Running hscolour for HTTP-4000.0.9...
setup: /tmp/plist/HTTP/share/hscolour-1.15/hscolour.css: copyFile: does not exist (No such file or directory)
*** Error code 1

Stop in /usr/ports/www/hs-HTTP.
*** Error code 1

Stop in /usr/ports/www/hs-HTTP.
       21.26 real        15.46 user         1.20 sys

What happened? genplist is working in its own sandbox in /tmp/plists. In our Makefile we specified the path to the hscolour css-file as ${PREFIX}/share/hscolour-1.15. However PREFIX has been set to /tmp/plists. I haven't really found a good solution to this. As a workaround, I will temporarily replace that path in our Makefile with the absolute path /usr/local/share/hscolour-1.15. I also have to delete the contents of /tmp/plists directory before I can start over.

#rm -r /tmp/plists/* && genplist create /tmp/plists

...

ocumentation created: dist/doc/html/HTTP/index.html
===>  Installing for hs-HTTP-4000.0.9
===>   hs-HTTP-4000.0.9 depends on executable: ghc - found
===>   Generating temporary packing list
===>  Checking if www/hs-HTTP already installed
Installing library in /tmp/plists/HTTP/lib/HTTP-4000.0.9/ghc-6.10.4
Registering HTTP-4000.0.9...
Reading package info from "dist/installed-pkg-config" ... done.
Writing new package config file... done.
===>   Registering installation for hs-HTTP-4000.0.9
       28.10 real        18.10 user         2.57 sys

This time it worked. A new file pkg-plist.new has been created. It contains all the files created by the port.

#vim pkg-plist.new

lib/HTTP-4000.0.9/ghc-6.10.4/HSHTTP-4000.0.9.o
lib/HTTP-4000.0.9/ghc-6.10.4/Network/Browser.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/BufferType.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/Auth.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/Base.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/Base64.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/Cookie.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/HandleStream.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/Headers.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/MD5.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/MD5Aux.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/Proxy.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/Stream.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP/Utils.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/Stream.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/StreamDebugger.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/StreamSocket.hi
lib/HTTP-4000.0.9/ghc-6.10.4/Network/TCP.hi
lib/HTTP-4000.0.9/ghc-6.10.4/libHSHTTP-4000.0.9.a
lib/HTTP-4000.0.9/register.sh
%%PORTDOCS%%%%DOCSDIR%%-4000.0.9/LICENSE
...

... more %%PORTDOCS%% stuff we can ignore

...

%%PORTDOCS%%%%DOCSDIR%%-4000.0.9/html/src/hscolour.css
%%PORTDOCS%%@dirrm %%DOCSDIR%%-4000.0.9/html/src
%%PORTDOCS%%@dirrm %%DOCSDIR%%-4000.0.9/html
%%PORTDOCS%%@dirrm %%DOCSDIR%%-4000.0.9
@dirrm lib/HTTP-4000.0.9/ghc-6.10.4/Network/HTTP
@dirrm lib/HTTP-4000.0.9/ghc-6.10.4/Network
@dirrm lib/HTTP-4000.0.9/ghc-6.10.4
@dirrm lib/HTTP-4000.0.9

We've already dealt with the PORTDOCS, so we can safely remove all those lines. We could keep the remaining lines - everything in lib/. However, imagine that the GHC version is updated to 6.12.1 in the near future. Then we would have to not only have to adjust the Makefile, but the pkg-plist as well. That would be much more complicated than necessary. So we also define plist substitutions for the LIBDIR_REL and GHC_VERSION in the Makefile.

PLIST_SUB+= LIBDIREL=lib/HTTP-4000.0.9

PLIST_SUB+= GHC_VERSION=6.10.4

Then we can rewrite the lines. For example the line:

lib/HTTP-4000.0.9/ghc-6.10.4/Network/Browser.hi

can be written as:

%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/Browser.hi

This way we can leave the pkg-plist untouched if the version of GHC or HTTP were to change. That is much more convenient. The complete pkg-plist looks like this:

%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/HSHTTP-%%HTTP_VERSION%%.o
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/Browser.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/BufferType.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/Auth.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/Base.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/Base64.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/Cookie.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/HandleStream.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/Headers.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/MD5.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/MD5Aux.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/Proxy.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/Stream.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP/Utils.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/Stream.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/StreamDebugger.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/StreamSocket.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/TCP.hi
%%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/libHSHTTP-%%HTTP_VERSION%%.a
%%HTTP_LIBDIR_REL%%/register.sh
%%NOPORTDOCS%%%%DOCSDIR%%/LICENSE
%%NOPORTDOCS%%@dirrmtry %%DOCSDIR%%
@dirrm %%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network/HTTP
@dirrm %%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%/Network
@dirrm %%HTTP_LIBDIR_REL%%/ghc-%%GHC_VERSION%%
@dirrm %%HTTP_LIBDIR_REL%%
@exec /bin/sh %D/%%HTTP_LIBDIR_REL%%/register.sh
@exec /bin/rm -f %D/lib/ghc-%%GHC_VERSION%%/package.conf.old
@unexec %D/bin/ghc-pkg unregister HTTP
@unexec /bin/rm -f %D/lib/ghc-%%GHC_VERSION%%/package.conf.old

To copy the pkg-plist.new to pkg-plist do commit.

#genplist commit

To test whether your pkg-plist really covers everything you can test it with.

#genplist test

If everything checks out, you can clean up.

#genplist clean

Testing the Port

At this point our port is complete. Time to test it.

#port test

===> Validating port with portlint
WARN: Makefile: [70]: possible direct use of command "install" found. use ${INSTALL_foobaa} instead.
WARN: Makefile: possible use of absolute pathname "/usr/local/share/hsc...".
WARN: Makefile: possible direct use of "/usr/local" found. if so, use ${PREFIX} or ${LOCALBASE}, as appropriate.
WARN: Makefile: only one MASTER_SITE configured.  Consider adding additional mirrors.0 fatal errors and 4 warnings found.
===> flags: PREFIX=/tmp/hs-HTTP-4000.0.9 NO_DEPENDS=yes PKG_DBDIR=/tmp/pkg_db.AkQQPcPV --
===> Cleaning workspace before port test
===>  Cleaning for hs-HTTP-4000.0.9
===>  Vulnerability check disabled, database not found
===>  Extracting for hs-HTTP-4000.0.9
=> MD5 Checksum OK for HTTP-4000.0.9.tar.gz.
=> SHA256 Checksum OK for HTTP-4000.0.9.tar.gz.
===>  Patching for hs-HTTP-4000.0.9
===>  Configuring for hs-HTTP-4000.0.9
[1 of 1] Compiling Main             ( Setup.lhs, Setup.o )
Linking setup ...
Configuring HTTP-4000.0.9...
...

...
===>  Installing for hs-HTTP-4000.0.9
===>   Generating temporary packing list
===>  Checking if www/HTTP already installed
Installing library in /tmp/hs-HTTP-4000.0.9/lib/HTTP-4000.0.9/ghc-6.10.4
Registering HTTP-4000.0.9...
Reading package info from "dist/installed-pkg-config" ... done.
Writing new package config file... done.
===>   Registering installation for hs-HTTP-4000.0.9
===>  Building package for hs-HTTP-4000.0.9
Creating package /usr/ports/packages/All/hs-HTTP-4000.0.9.tbz
Registering depends:.
Creating bzip'd tar ball in '/usr/ports/packages/All/hs-HTTP-4000.0.9.tbz'
===> Checking pkg_info
hs-HTTP-4000.0.9    A library for client-side HTTP
===> Checking shared library dependencies
===>  Deinstalling for www/HTTP
===>   Deinstalling hs-HTTP-4000.0.9
/tmp/hs-HTTP-4000.0.9/bin/ghc-pkg: not found
pkg_delete: unexec command for '/tmp/hs-HTTP-4000.0.9/bin/ghc-pkg unregister HTTP' failed
pkg_delete: couldn't entirely delete package (perhaps the packing list is
incorrectly specified?)
===> Extra files and directories check
===> Cleaning up after port test
===>  Cleaning for hs-HTTP-4000.0.9
===>  Removing existing /tmp/hs-HTTP-4000.0.9 dir
===> Done.

I've highlighted the important messages. The first three warnings come from portlint who is reminding us that we used an explicit path to hscolor.css for testing. Now is probably a good time to revert that change.

The other warning was generated because ghc-pkg was not found in the the /tmp/hs-HTTP directory. That's not very surprising either. For a "real" port unregistering will work.

Portlint also suggested specifying more than a one source in MASTER_SITE, which is good idea, but there doesn't seem to exist another one.

Submitting the Port

If all tests passed without errors, the last step is to submit the port. Before you do that make sure you read DOs and DON'Ts from the porters handbook to safe yourself and a potential committer a lot of headaches.

#port submit

This will fire up a problem report (PR) editor. The most important values have already been set and our new port has already been attached as shar-archive. When you close the editor, it asks you what to do with the report. Hit 's' to submit it.

s)end, e)dit or a)bort? s

After some time the report will be listed in GNATS and added to the portstree by one of the FreeBSD committers. If there are problems, you will be contacted. That's all there is to it : )

Porting Cabal packages to FreeBSD - Part 1

It seems that all the Haskell ports on freshports have been committed by only a handful of people. This is unfortunate since porting a cabal-package to FreeBSD is almost as easy as creating it in the first place. There are some manual steps involved, but overall it's fairly simple once you have a basic understanding of cabal-packages and the FreeBSD ports. The FreeBSD side of things is a little bit more complicated, but it is also excellently documented in the Porter's Handbook. In this post I will try to give an overview of BSD's port system, what's necessary to create a port and mention a couple of tools that help creating new ports. Hopefully it will motivate some people to join the FreeBSD maintainers.

The 1st Part is intended as quick introduction. I will assume that you have very little knowledge about the BSD Ports, but I will also expect that you consult the handbook sooner rather than later. If you have questions that are not covered in the handbook, you can ask on the freebsd-porters mailing list. In the second part I will walk you through the complete process of porting a Haskell cabal package.

Cabal Package System

As you probably know, Haskell uses it's very own cabal package system. It does everything you expect from a package management system (and a little more). As porters we can't rely on cabal, however, and have to manually install cabal packages (see the cabal manual for more information) through BSD's ports so that the system is aware of the installed software. But before jumping straight to BSD, let's first look at how you manually install a cabal-package. All you need are these commands and you're done.

• setup configure
• setup build 
• setup register
  
• setup install 

Now that is simple, isn't it? Theoretically, these commands are sufficient. In reality, the configure step will fail if there are some unresolved dependencies to other packages as often will be the case. Why am I even bothering you with all of this? As you will see later, we are going to use these steps at the core of the Makefile of our port. Issuing these commands from within our Makefile is gonna be as complicated as it gets. But I'm getting ahead of myself again. Next I'll give you the 10'000 feet overview of the FreeBSD ports. 

Anatomy of a FreeBSD Port

All ports are located in one of the sub directories of /usr/ports. Where exactly is determined by which category the port belongs to. A typical FreeBSD port consists of four files:

pkg-descr

This file contains a description of the port, what it does and so on. Typically that will be one or two paragraphs.

pkg-plist

The packing list contains a listing of files that will be installed as part of the port. This is important, e.g. to cleanly deinstall a port. Not all files have to be added explicitly to pkg-plist, however. Man pages are a notable exception - those have to be declared in the Makefile. Generally, I find keeping track of these files a bit tedious, but fortunately there are tools that help you generate the packing list automatically.

distinfo

This file contains md5 and SHA checksums to verify the data integrity of the port. It too, can be comfortably auto-generated by typing make makesum.

Makefile

The Port's Makefile does all the heavy lifting. Make is a relatively old utility for automatically building executable programs and libraries from source code. Normally a makefile defines a set of operations it supports so called targets. They're used to download the source tarballs, check their checksums, resolve any dependencies to other ports and compile and install everything. Don't worry, most of these functions will be imported from the master file bsd.port.mk. As Haskell porters we only have to fill in a couple of blanks in form of control variables and a couple of targets.

Many of the variables you need to provide are self-explanatory (all of them are covered in more detail in chapter 5 of the Porter's Handbook so I won't duplicate everything here)

• PORTNAME: contains the name under which the port will be installed

• PORTVERSION: contains the version number of the port.

• PORTREVISION: The PORTREVISION variable is a monotonically increasing value which is reset to 0 with every increase of PORTVERSION (i.e. every time a new official vendor release is made), and appended to the package name if non-zero.

• CATEGORIES: contains a list of categories the port belongs too. This list also determines where in the ports tree the port is imported. If you put more than one category here, it is assumed that the port files will be put in the sub directory with the name in the first category. Please have a look at Chapter 5.3 of the Porter's Handbook, which explains the port categorization in detail.
  

• MASTER_SITES: contains a list of locations of the source tarball. Specifying more than one site is recommended for redundancy reasons.

• PKGNAMEPREFIX: The prefix for all Haskell ports is 'hs-'.
  

• MAINTAINER: Your email address.
  

• COMMENT: This is a one-line description of the port. Please do not include the package name (or version number of the software) in the comment. The comment should begin with a capital and end without a period.

• LIB_DEPENDS: This variable specifies the shared libraries this port depends on.

• RUN_DEPENDS: This variable specifies executables or files this port depends on during run-time. GHC is a typical run dependency.
  

• BUILD_DEPENDS: You guessed it. This variable specifies executables or files this port requires to build. GHC is also a typical build dependency.

This concludes the broad overview of the most important variables. Of course, as maintainer you can also define your own variables for convenience. We'll see some of this in the next paragraph. So let's move on to the interesting stuff. The custom build actions. They're executed by the ports system in the order they're listed below.

• do-configure: The configuration step. The first step is to actually build the setup binary. If that succeeds, setup configure will be called just as in the manual installation.
  

• do-build: setup build is called after successful configuration. Additionally, generate a script to register the new package with the Haskell compiler. At this stage, we can also generate HTML documentation through haddock.
  

• do-install: the next step is to call setup install. Besides that, we also copy the register.sh script that was created before into directory that contains all the compiled sources. This is necessary so that you can register the libraries when installing a pre-compiled *.tbz package.
  

• post-install: We use the post-installation stage to clean up a temporarily file that was created outside our working directory. Namely package.conf.old

Most (if not all) Haskell ports I've seen in the ports define these four targets.

For example here are the four target from the Makefile of archivers/hs-zlib:

do-configure:
        cd ${WRKSRC} && ${GHC_CMD} --make Setup.lhs -o setup -package Cabal \
                                 && ${SETUP_CMD} configure --haddock-options=-w --prefix=${PREFIX}

do-build:
        cd ${WRKSRC} && ${SETUP_CMD} build \
                                  && ${SETUP_CMD} register --gen-script

.if !defined(NOPORTDOCS)
        cd ${WRKSRC} && ${SETUP_CMD} haddock --hyperlink-source \
                                          --hscolour-css=${HSCOLOUR_DATADIR}/hscolour.css
.endif

do-install:
        cd ${WRKSRC} && ${SETUP_CMD} install \
                                  && ${INSTALL_SCRIPT} register.sh ${PREFIX}/${SOME_LIBDIR_REL}/register.sh

post-install:
        ${RM} -f ${PREFIX}/lib/ghc-${GHC_VERSION}/package.conf.old

The only thing that changed compared to the manual installation process outlined above is that the maintainer defined some variables for more flexibility. Rather than ./setup, ${SETUP} is used the same was done for ${GHC_CMD}.

The Toolbox

There are also a couple of tools I previously alluded to that help you create and test new ports:

• porttools: ports-mgmt/porttools provides tools for testing and submitting port updates and new ports.
• genplist: ports-mgmt/genplist automatically creates a static plist for a port by installing it into a temporary directory, and then examining the directory tree. The process is based on the instructions for plist generation in the FreeBSD Porter's Handbook.
• portlint: Please use portlint to see if your port conforms to our guidelines. The ports-mgmt/portlint program is part of the ports collection. In particular, you may want to check if the Makefile is in the right shape and the package is named appropriately

I'll demonstrate how they work in part 2.