awful debugging hacking

[dpkg] / man / dpkg-gensymbols.man

.\" dpkg manual page - dpkg-gensymbols(1)
.\"
.\" Copyright © 2007-2011 Raphaël Hertzog <hertzog@debian.org>
.\" Copyright © 2009-2010 Modestas Vainius <modestas@vainius.eu>
.\" Copyright © 2012-2015 Guillem Jover <guillem@debian.org>
.\"
.\" This is free software; you can redistribute it and/or modify
.\" it under the terms of the GNU General Public License as published by
.\" the Free Software Foundation; either version 2 of the License, or
.\" (at your option) any later version.
.\"
.\" This is distributed in the hope that it will be useful,
.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
.\" GNU General Public License for more details.
.\"
.\" You should have received a copy of the GNU General Public License
.\" along with this program.  If not, see <https://www.gnu.org/licenses/>.
.
.TH dpkg\-gensymbols 1 "%RELEASE_DATE%" "%VERSION%" "dpkg suite"
.nh
.SH NAME
dpkg\-gensymbols \- generate symbols files (shared library dependency information)
.
.SH SYNOPSIS
.B dpkg\-gensymbols
.RI [ option ...]
.
.SH DESCRIPTION
.B dpkg\-gensymbols
scans a temporary build tree (debian/tmp by default) looking for libraries
and generates a \fIsymbols\fR file describing them. This file, if
non-empty, is then installed in the DEBIAN subdirectory of the build tree
so that it ends up included in the control information of the package.
.P
When generating those files, it uses as input some symbols files
provided by the maintainer. It looks for the following files (and uses the
first that is found):
.IP • 4
debian/\fIpackage\fR.symbols.\fIarch\fR
.IP • 4
debian/symbols.\fIarch\fR
.IP • 4
debian/\fIpackage\fR.symbols
.IP • 4
debian/symbols
.P
The main interest of those files is to provide the minimal version
associated to each symbol provided by the libraries. Usually it
corresponds to the first version of that package that provided the symbol,
but it can be manually incremented by the maintainer if the ABI of the
symbol is extended without breaking backwards compatibility. It's the
responsibility of the maintainer to keep those files up-to-date and
accurate, but \fBdpkg\-gensymbols\fR helps with that.
.P
When the generated symbols files differ from the maintainer supplied
one, \fBdpkg\-gensymbols\fR will print a diff between the two versions.
Furthermore if the difference is too significant, it will even fail (you
can customize how much difference you can tolerate, see the \fB\-c\fR
option).
.SH MAINTAINING SYMBOLS FILES
The symbols files are really useful only if they reflect the evolution of
the package through several releases. Thus the maintainer has to update
them every time that a new symbol is added so that its associated minimal
version matches reality.
The diffs contained in the build logs can be used as a starting point,
but the maintainer, additionally, has to make sure that the behaviour
of those symbols has not changed in a way that would make anything
using those symbols and linking against the new version, stop working
with the old version.
In most cases, the diff applies directly to the
debian/\fIpackage\fR.symbols file. That said, further tweaks are usually
needed: it's recommended for example to drop the Debian revision
from the minimal version so that backports with a lower version number
but the same upstream version still satisfy the generated dependencies.
If the Debian revision can't be dropped because the symbol really got
added by the Debian specific change, then one should suffix the version
with ‘\fB~\fP’.
.P
Before applying any patch to the symbols file, the maintainer should
double-check that it's sane. Public symbols are not supposed to disappear,
so the patch should ideally only add new lines.
.P
Note that you can put comments in symbols files: any line with ‘#’ as
the first character is a comment except if it starts with ‘#include’
(see section \fBUsing includes\fP).
Lines starting with ‘#MISSING:’ are special comments documenting
symbols that have disappeared.
.P
Do not forget to check if old symbol versions need to be increased.
There is no way \fBdpkg\-gensymbols\fP can warn about this. Blindly
applying the diff or assuming there is nothing to change if there is
no diff, without checking for such changes, can lead to packages with
loose dependencies that claim they can work with older packages they
cannot work with. This will introduce hard to find bugs with (partial)
upgrades.
.SS Using #PACKAGE# substitution
.P
In some rare cases, the name of the library varies between architectures.
To avoid hardcoding the name of the package in the symbols file, you can
use the marker \fI#PACKAGE#\fR. It will be replaced by the real package
name during installation of the symbols files. Contrary to the
\fI#MINVER#\fR marker, \fI#PACKAGE#\fR will never appear in a symbols file
inside a binary package.
.SS Using symbol tags
.P
Symbol tagging is useful for marking symbols that are special in some way.  Any
symbol can have an arbitrary number of tags associated with it. While all tags are
parsed and stored, only some of them are understood by
\fBdpkg\-gensymbols\fR and trigger special handling of the symbols. See
subsection \fBStandard symbol tags\fR for reference of these tags.
.P
Tag specification comes right before the symbol name (no whitespace is allowed
in between). It always starts with an opening bracket \fB(\fR, ends with a
closing bracket \fB)\fR and must contain at least one tag. Multiple tags are
separated by the \fB|\fR character. Each tag can optionally have a value which
is separated form the tag name by the \fB=\fR character. Tag names and values
can be arbitrary strings except they cannot contain any of the special \fB)\fR
\fB|\fR \fB=\fR characters. Symbol names following a tag specification can
optionally be quoted with either \fB'\fR or \fB"\fR characters to allow
whitespaces in them. However, if there are no tags specified for the symbol,
quotes are treated as part of the symbol name which continues up until the
first space.
.P
 (tag1=i am marked|tag name with space)"tagged quoted symbol"@Base 1.0
 (optional)tagged_unquoted_symbol@Base 1.0 1
 untagged_symbol@Base 1.0
.P
The first symbol in the example is named \fItagged quoted symbol\fR and has two
tags: \fItag1\fR with value \fIi am marked\fR and \fItag name with space\fR
that has no value. The second symbol named \fItagged_unquoted_symbol\fR is
only tagged with the tag named \fIoptional\fR. The last symbol is an
example of the normal untagged symbol.
.P
Since symbol tags are an extension of the \fBdeb\-symbols\fP(5) format, they
can only be part of the symbols files used in source packages (those files
should then be seen as templates used to build the symbols files that are
embedded in binary packages). When
\fBdpkg\-gensymbols\fR is called without the \fB\-t\fP option, it will
output symbols files compatible to the \fBdeb\-symbols\fP(5) format:
it fully processes symbols according to the requirements of their standard tags
and strips all tags from the output. On the contrary, in template mode
(\fB\-t\fP) all symbols and their tags (both standard and unknown ones)
are kept in the output and are written in their original form as they were
loaded.
.SS Standard symbol tags
.TP
.B optional
A symbol marked as optional can disappear from the library at any time and that
will never cause \fBdpkg\-gensymbols\fR to fail. However, disappeared optional
symbols will continuously appear as MISSING in the diff in each new package
revision.  This behaviour serves as a reminder for the maintainer that such a
symbol needs to be removed from the symbol file or readded to the library. When
the optional symbol, which was previously declared as MISSING, suddenly
reappears in the next revision, it will be upgraded back to the “existing”
status with its minimum version unchanged.

This tag is useful for symbols which are private where their disappearance do
not cause ABI breakage. For example, most of C++ template instantiations fall
into this category. Like any other tag, this one may also have an arbitrary
value: it could be used to indicate why the symbol is considered optional.
.TP
.B arch=\fIarchitecture-list\fR
.TQ
.B arch\-bits=\fIarchitecture-bits\fR
.TQ
.B arch\-endian=\fIarchitecture-endianness\fR
These tags allow one to restrict the set of architectures where the symbol
is supposed to exist. The \fBarch\-bits\fP and \fBarch\-endian\fP tags
are supported since dpkg 1.18.0. When the symbols list is updated with
the symbols
discovered in the library, all arch-specific symbols which do not concern
the current host architecture are treated as if they did not exist. If an
arch-specific symbol matching the current host architecture does not exist
in the library, normal procedures for missing symbols apply and it may
cause \fBdpkg\-gensymbols\fR to fail. On the other hand, if the
arch-specific symbol is found when it was not supposed to exist (because
the current host architecture is not listed in the tag or does not match
the endianness and bits), it is made arch neutral (i.e. the arch, arch-bits
and arch-endian tags are dropped and the symbol will appear in the diff due
to this change), but it is not considered as new.

When operating in the default non-template mode, among arch-specific symbols
only those that match the current host architecture are written to the
symbols file. On the contrary, all arch-specific symbols (including those
from foreign arches) are always written to the symbol file when operating
in template mode.

The format of \fIarchitecture-list\fR is the same as the one used in the
\fBBuild\-Depends\fP field of \fIdebian/control\fR (except the enclosing
square brackets []). For example, the first symbol from the list below
will be considered only on alpha, any\-amd64 and ia64 architectures,
the second only on linux architectures, while the third one anywhere
except on armel.

 (arch=alpha any\-amd64 ia64)64bit_specific_symbol@Base 1.0
 (arch=linux\-any)linux_specific_symbol@Base 1.0
 (arch=!armel)symbol_armel_does_not_have@Base 1.0

The \fIarchitecture-bits\fP is either \fB32\fP or \fB64\fP.

 (arch-bits=32)32bit_specific_symbol@Base 1.0
 (arch-bits=64)64bit_specific_symbol@Base 1.0

The \fIarchitecture-endianness\fP is either \fBlittle\fP or \fBbig\fP.

 (arch-endian=little)little_endian_specific_symbol@Base 1.0
 (arch-endian=big)big_endian_specific_symbol@Base 1.0

Multiple restrictions can be chained.

 (arch-bits=32|arch-endian=little)32bit_le_symbol@Base 1.0
.TP
.B ignore\-blacklist
dpkg\-gensymbols has an internal blacklist of symbols that should not
appear in symbols files as they are usually only side-effects of
implementation details of the toolchain. If for some reason, you really
want one of those symbols to be included in the symbols file, you should
tag the symbol with \fBignore\-blacklist\fP. It can be necessary for
some low level toolchain libraries like libgcc.
.TP
.B c++
Denotes \fIc++\fR symbol pattern. See \fBUsing symbol patterns\fR subsection
below.
.TP
.B symver
Denotes \fIsymver\fR (symbol version) symbol pattern. See \fBUsing symbol
patterns\fR subsection below.
.TP
.B regex
Denotes \fIregex\fR symbol pattern. See \fBUsing symbol patterns\fR subsection
below.
.SS Using symbol patterns
.P
Unlike a standard symbol specification, a pattern may cover multiple real
symbols from the library. \fBdpkg\-gensymbols\fR will attempt to match each
pattern against each real symbol that does \fInot\fR have a specific symbol
counterpart defined in the symbol file. Whenever the first matching pattern is
found, all its tags and properties will be used as a basis specification of the
symbol. If none of the patterns matches, the symbol will be considered as new.

A pattern is considered lost if it does not match any symbol in the library. By
default this will trigger a \fBdpkg\-gensymbols\fP failure under \fB\-c1\fP or
higher level. However, if the failure is undesired, the pattern may be marked
with the \fIoptional\fR tag. Then if the pattern does not match anything, it
will only appear in the diff as MISSING. Moreover, like any symbol, the pattern
may be limited to the specific architectures with the \fIarch\fR tag. Please
refer to \fBStandard symbol tags\fR subsection above for more information.

Patterns are an extension of the \fBdeb\-symbols\fP(5) format hence they are
only valid in symbol file templates. Pattern specification syntax is not any
different from the one of a specific symbol. However, symbol name part of the
specification serves as an expression to be matched against \fIname@version\fR
of the real symbol. In order to distinguish among different pattern types, a
pattern will typically be tagged with a special tag.

At the moment, \fBdpkg\-gensymbols\fR supports three basic pattern types:
.TP 3
.B c++
This pattern is denoted by the \fIc++\fR tag. It matches only C++ symbols by
their demangled symbol name (as emitted by \fBc++filt\fR(1) utility). This
pattern is very handy for matching symbols which mangled names might vary
across different architectures while their demangled names remain the same. One
group of such symbols is \fInon\-virtual thunks\fR which have architecture
specific offsets embedded in their mangled names. A common instance of this
case is a virtual destructor which under diamond inheritance needs a
non-virtual thunk symbol. For example, even if _ZThn8_N3NSB6ClassDD1Ev@Base on
32bit architectures will probably be _ZThn16_N3NSB6ClassDD1Ev@Base on 64bit
ones, it can be matched with a single \fIc++\fR pattern:

libdummy.so.1 libdummy1 #MINVER#
 [...]
 (c++)"non\-virtual thunk to NSB::ClassD::~ClassD()@Base" 1.0
 [...]

The demangled name above can be obtained by executing the following command:

 $ echo '_ZThn8_N3NSB6ClassDD1Ev@Base' | c++filt

Please note that while mangled name is unique in the library by definition,
this is not necessarily true for demangled names. A couple of distinct real
symbols may have the same demangled name. For example, that's the case with
non-virtual thunk symbols in complex inheritance configurations or with most
constructors and destructors (since g++ typically generates two real symbols
for them). However, as these collisions happen on the ABI level, they should
not degrade quality of the symbol file.
.TP
.B symver
This pattern is denoted by the \fIsymver\fR tag. Well maintained libraries have
versioned symbols where each version corresponds to the upstream version where
the symbol got added. If that's the case, you can use a \fIsymver\fR pattern to
match any symbol associated to the specific version. For example:

libc.so.6 libc6 #MINVER#
 (symver)GLIBC_2.0 2.0
 [...]
 (symver)GLIBC_2.7 2.7
 access@GLIBC_2.0 2.2

All symbols associated with versions GLIBC_2.0 and GLIBC_2.7 will lead to
minimal version of 2.0 and 2.7 respectively with the exception of the symbol
access@GLIBC_2.0. The latter will lead to a minimal dependency on libc6 version
2.2 despite being in the scope of the "(symver)GLIBC_2.0" pattern because
specific symbols take precedence over patterns.

Please note that while old style wildcard patterns (denoted by "*@version" in
the symbol name field) are still supported, they have been deprecated by new
style syntax "(symver|optional)version". For example, "*@GLIBC_2.0 2.0" should
be written as "(symver|optional)GLIBC_2.0 2.0" if the same behaviour is needed.
.TP
.B regex
Regular expression patterns are denoted by the \fIregex\fR tag. They match by
the perl regular expression specified in the symbol name field. A regular
expression is matched as it is, therefore do not forget to start it with the
\fI^\fR character or it may match any part of the real symbol
\fIname@version\fR string. For example:

libdummy.so.1 libdummy1 #MINVER#
 (regex)"^mystack_.*@Base$" 1.0
 (regex|optional)"private" 1.0

Symbols like "mystack_new@Base", "mystack_push@Base", "mystack_pop@Base" etc.
will be matched by the first pattern while e.g. "ng_mystack_new@Base" won't.
The second pattern will match all symbols having the string "private" in their
names and matches will inherit \fIoptional\fR tag from the pattern.
.P
Basic patterns listed above can be combined where it makes sense. In that case,
they are processed in the order in which the tags are specified. For example,
both

 (c++|regex)"^NSA::ClassA::Private::privmethod\\d\\(int\\)@Base" 1.0
 (regex|c++)N3NSA6ClassA7Private11privmethod\\dEi@Base 1.0

will match symbols "_ZN3NSA6ClassA7Private11privmethod1Ei@Base" and
"_ZN3NSA6ClassA7Private11privmethod2Ei@Base". When matching the first pattern,
the raw symbol is first demangled as C++ symbol, then the demangled name is
matched against the regular expression. On the other hand, when matching the
second pattern, regular expression is matched against the raw symbol name, then
the symbol is tested if it is C++ one by attempting to demangle it. A failure
of any basic pattern will result in the failure of the whole pattern.
Therefore, for example, "__N3NSA6ClassA7Private11privmethod\\dEi@Base" will not
match either of the patterns because it is not a valid C++ symbol.

In general, all patterns are divided into two groups: aliases (basic \fIc++\fR
and \fIsymver\fR) and generic patterns (\fIregex\fR, all combinations of
multiple basic patterns). Matching of basic alias-based patterns is fast (O(1))
while generic patterns are O(N) (N - generic pattern count) for each symbol.
Therefore, it is recommended not to overuse generic patterns.

When multiple patterns match the same real symbol, aliases (first \fIc++\fR,
then \fIsymver\fR) are preferred over generic patterns. Generic patterns are
matched in the order they are found in the symbol file template until the first
success.  Please note, however, that manual reordering of template file entries
is not recommended because \fBdpkg\-gensymbols\fR generates diffs based on the
alphanumerical order of their names.
.SS Using includes
.P
When the set of exported symbols differ between architectures, it may become
inefficient to use a single symbol file. In those cases, an include directive
may prove to be useful in a couple of ways:
.IP • 4
You can factorize the common part in some external file
and include that file in your \fIpackage\fR.symbols.\fIarch\fR file by
using an include directive like this:

#include "\fIpackages\fR.symbols.common"
.IP •
The include directive may also be tagged like any symbol:

(tag|...|tagN)#include "file-to-include"

As a result, all symbols included from \fIfile-to-include\fR will be considered
to be tagged with \fItag\fR ... \fItagN\fR by default. You can use this feature
to create a common \fIpackage\fR.symbols file which includes architecture
specific symbol files:

  common_symbol1@Base 1.0
 (arch=amd64 ia64 alpha)#include "package.symbols.64bit"
 (arch=!amd64 !ia64 !alpha)#include "package.symbols.32bit"
  common_symbol2@Base 1.0
.P
The symbols files are read line by line, and include directives are processed
as soon as they are encountered. This means that the content of the included
file can override any content that appeared before the include directive and
that any content after the directive can override anything contained in the
included file. Any symbol (or even another #include directive) in the included
file can specify additional tags or override values of the inherited tags in
its tag specification. However, there is no way for the symbol to remove
any of the inherited tags.
.P
An included file can repeat the header line containing the SONAME of the
library. In that case, it overrides any header line previously read.
However, in general it's best to avoid duplicating header lines. One way
to do it is the following:
.PP
#include "libsomething1.symbols.common"
 arch_specific_symbol@Base 1.0
.SS Good library management
.P
A well-maintained library has the following features:
.IP • 4
its API is stable (public symbols are never dropped, only new public
symbols are added) and changes in incompatible ways only when the SONAME
changes;
.IP • 4
ideally, it uses symbol versioning to achieve ABI stability despite
internal changes and API extension;
.IP • 4
it doesn't export private symbols (such symbols can be tagged optional as
workaround).
.P
While maintaining the symbols file, it's easy to notice appearance and
disappearance of symbols. But it's more difficult to catch incompatible
API and ABI change. Thus the maintainer should read thoroughly the
upstream changelog looking for cases where the rules of good library
management have been broken. If potential problems are discovered,
the upstream author should be notified as an upstream fix is always better
than a Debian specific work-around.
.SH OPTIONS
.TP
.BI \-P package-build-dir
Scan \fIpackage-build-dir\fR instead of debian/tmp.
.TP
.BI \-p package
Define the package name. Required if more than one binary package is listed in
debian/control (or if there's no debian/control file).
.TP
.BI \-v version
Define the package version. Defaults to the version extracted from
debian/changelog. Required if called outside of a source package tree.
.TP
.BI \-e library-file
Only analyze libraries explicitly listed instead of finding all public
libraries. You can use shell patterns used for pathname expansions (see
the \fBFile::Glob\fP(3perl) manual page for details) in \fIlibrary-file\fR
to match multiple libraries with a single argument (otherwise you need
multiple \fB\-e\fR).
.TP
.BI \-I filename
Use \fIfilename\fR as reference file to generate the symbols file
that is integrated in the package itself.
.TP
.BR \-O [\fIfilename\fP]
Print the generated symbols file to standard output or to \fIfilename\fR
if specified, rather than to
.B debian/tmp/DEBIAN/symbols
(or
.IB package-build-dir /DEBIAN/symbols
if
.B \-P
was used). If \fIfilename\fR is pre-existing, its contents are used as
basis for the generated symbols file.
You can use this feature to update a symbols file so that it matches a
newer upstream version of your library.
.TP
.BI \-t
Write the symbol file in template mode rather than the format compatible with
\fBdeb\-symbols\fP(5). The main difference is that in the template mode symbol
names and tags are written in their original form contrary to the
post-processed symbol names with tags stripped in the compatibility mode.
Moreover, some symbols might be omitted when writing a standard
\fBdeb\-symbols\fP(5) file (according to the tag processing rules) while all
symbols are always written to the symbol file template.
.TP
.BI \-c [0-4]
Define the checks to do when comparing the generated symbols file with the
template file used as starting point. By default the level is 1. Increasing
levels do more checks and include all checks of lower levels. Level 0 never
fails. Level 1 fails if some symbols have disappeared. Level 2 fails if some
new symbols have been introduced. Level 3 fails if some libraries have
disappeared. Level 4 fails if some libraries have been introduced.

This value can be overridden by the environment variable
.BR DPKG_GENSYMBOLS_CHECK_LEVEL .
.TP
.BI \-q
Keep quiet and never generate a diff between generated symbols file and the
template file used as starting point or show any warnings about new/lost
libraries or new/lost symbols. This option only disables informational output
but not the checks themselves (see \fB\-c\fP option).
.TP
.BI \-a arch
Assume \fIarch\fR as host architecture when processing symbol files. Use this
option to generate a symbol file or diff for any architecture provided its
binaries are already available.
.TP
.BI \-d
Enable debug mode. Numerous messages are displayed to explain what
.B dpkg\-gensymbols
does.
.TP
.BI \-V
Enable verbose mode. The generated symbols file contains deprecated
symbols as comments. Furthermore in template mode, pattern symbols
are followed by comments listing real symbols that have matched the
pattern.
.TP
.BR \-? ", " \-\-help
Show the usage message and exit.
.TP
.BR \-\-version
Show the version and exit.
.
.SH ENVIRONMENT
.TP
.B DPKG_GENSYMBOLS_CHECK_LEVEL
Overrides the command check level, even if the \fB\-c\fP command-line
argument was given (note that this goes against the common convention
of command-line arguments having precedence over environment variables).
.SH SEE ALSO
.BR https://people.redhat.com/drepper/symbol\-versioning
.br
.BR https://people.redhat.com/drepper/goodpractice.pdf
.br
.BR https://people.redhat.com/drepper/dsohowto.pdf
.br
.BR deb\-symbols (5),
.BR dpkg\-shlibdeps (1).