docs/develop/packages: Convert wiki files to MediaWiki syntax.

This is what GitHub actually is rendering pages as.
This commit is contained in:
Augustin Cavalier
2018-01-10 16:37:28 -05:00
parent 5492447a60
commit fc2b04e7e4
11 changed files with 629 additions and 438 deletions
+20 -19
View File
@@ -1,9 +1,10 @@
= Bootstrapping Haiku =
== Bootstrapping Haiku ==
Even a very basic Haiku requires a set of third-party packages (ICU, zlib,...), a Haiku sufficiently complete to build software even more (binutils, gcc, make,...). So whenever something fundamental in Haiku (the architecture ABI, the ABI of libroot) changes in a binary incompatible way, or when Haiku is ported to a new architecture, it is necessary to bootstrap Haiku and a basic set of required third-party packages. This document describes how this process works. Even a very basic Haiku requires a set of third-party packages (ICU, zlib,...), a Haiku sufficiently complete to build software even more (binutils, gcc, make,...). So whenever something fundamental in Haiku (the architecture ABI, the ABI of libroot) changes in a binary incompatible way, or when Haiku is ported to a new architecture, it is necessary to bootstrap Haiku and a basic set of required third-party packages. This document describes how this process works.
== Prerequisites == === Prerequisites ===
*So far the bootstrap build has only been tested on Linux. It will probably also work on FreeBSD and other Unixes. Haiku is not yet supported as a build platform. *So far the bootstrap build has only been tested on Linux. It will probably also work on FreeBSD and other Unixes. Haiku is not yet supported as a build platform.
*A required prerequisite for the bootstrap build is a checkout of the haikuporter, the haikuports, and the haikuports.cross repositories from the [https://github.com/haikuports/ Github HaikuPorts site]. *A required prerequisite for the bootstrap build is a checkout of the haikuporter, the haikuports, and the haikuports.cross repositories from the [https://github.com/haikuports/ Github HaikuPorts site].
@@ -15,48 +16,48 @@ Even a very basic Haiku requires a set of third-party packages (ICU, zlib,...),
*All the usual prerequisites for building Haiku. *All the usual prerequisites for building Haiku.
== Configuring and Building == === Configuring and Building ===
1. Configure the Haiku build with all usual parameters, but add the `--bootstrap` option with its three parameters, the paths to the checked out haikuporter ''script'' (eg. `../haikuporter/haikuporter`), haikuports.cross, and haikuports repositories folders: #Configure the Haiku build with all usual parameters, but add the <tt>--bootstrap</tt> option with its three parameters, the paths to the checked out haikuporter ''script'' (eg. <tt>../haikuporter/haikuporter</tt>), haikuports.cross, and haikuports repositories folders:
{{{ <pre>
.../configure ... --bootstrap path/to/haikuporter path/to/haikuports.cross path/to/haikuports .../configure ... --bootstrap path/to/haikuporter path/to/haikuports.cross path/to/haikuports
}}} </pre>
It is important that you build outside the tree, as otherwise the build will fail! It is important that you build outside the tree, as otherwise the build will fail!
1. Build a bootstrap Haiku image: #Build a bootstrap Haiku image:
{{{ <pre>
jam -q @bootstrap-raw jam -q @bootstrap-raw
}}} </pre>
1. Boot the bootstrap Haiku (e.g. in a virtual machine), edit the file "/boot/home/haikuports/haikuports.config" -- entering your email address in the "PACKAGER" field -- and finally open a Terminal and build the third-party packages: #Boot the bootstrap Haiku (e.g. in a virtual machine), edit the file "/boot/home/haikuports/haikuports.config" -- entering your email address in the "PACKAGER" field -- and finally open a Terminal and build the third-party packages:
{{{ <pre>
cd haikuports cd haikuports
haikuporter --do-bootstrap haikuporter --do-bootstrap
}}} </pre>
In the "packages" subdirectory the built packages are collected. This is the initial set of packages for the HaikuPorts packages repository, plus the source packages the Haiku build system put in your generated directory under "objects/haiku/<arch>/packaging/repositories/HaikuPorts-sources-build/packages" (ignore the "rigged" source packages). With these packages a HaikuPorts package repository can be built and in turn a regular Haiku can be built using it. Further packages can then be built on the regular Haiku. In the "packages" subdirectory the built packages are collected. This is the initial set of packages for the HaikuPorts packages repository, plus the source packages the Haiku build system put in your generated directory under "objects/haiku/<arch>/packaging/repositories/HaikuPorts-sources-build/packages" (ignore the "rigged" source packages). With these packages a HaikuPorts package repository can be built and in turn a regular Haiku can be built using it. Further packages can then be built on the regular Haiku.
Further hints: Further hints:
*Of course, as usual, "-j<number>" can be passed to the jam building the bootstrap Haiku. Building the bootstrap third-party packages, which is part of this process, will take quite some time anyway. Since those packages are built sequentially, the jam variable "HAIKU_PORTER_CONCURRENT_JOBS" can be defined to the number of jobs that shall be used to build a package. *Of course, as usual, "-j<number>" can be passed to the jam building the bootstrap Haiku. Building the bootstrap third-party packages, which is part of this process, will take quite some time anyway. Since those packages are built sequentially, the jam variable "HAIKU_PORTER_CONCURRENT_JOBS" can be defined to the number of jobs that shall be used to build a package.
*Instead of "bootstrap-raw" the build profile "bootstrap-vmware" can be used as well. You can also define your own build profile, e.g. for building to a partition. As long as its name starts with "bootstrap-" that will result in a bootstrap Haiku. *Instead of "bootstrap-raw" the build profile "bootstrap-vmware" can be used as well. You can also define your own build profile, e.g. for building to a partition. As long as its name starts with "bootstrap-" that will result in a bootstrap Haiku.
* `haikuporter` also supports the "-j<number>" option to specify the number of jobs to use. Even on real hardware this step will nevertheless take a long time. *<tt>haikuporter</tt> also supports the "-j<number>" option to specify the number of jobs to use. Even on real hardware this step will nevertheless take a long time.
* The jam variable HAIKU_PORTER_EXTRA_OPTIONS can be defined to any options that should be passed to `haikuporter` (`--debug` is handy for showing python strack traces, for instance). *The jam variable HAIKU_PORTER_EXTRA_OPTIONS can be defined to any options that should be passed to <tt>haikuporter</tt> (<tt>--debug</tt> is handy for showing python strack traces, for instance).
== How it works == === How it works ===
Building the bootstrap Haiku image is in principle quite similar to building a regular Haiku image, save for the following differences: Building the bootstrap Haiku image is in principle quite similar to building a regular Haiku image, save for the following differences:
*Some parts of a regular Haiku that aren't needed for building packages are omitted (e.g. the Demos, MediaPlayer, the OpenGL API,...). *Some parts of a regular Haiku that aren't needed for building packages are omitted (e.g. the Demos, MediaPlayer, the OpenGL API,...).
*Certain third-party packages that aren't needed for building packages are omitted as well. *Certain third-party packages that aren't needed for building packages are omitted as well.
* The third-party packages are not downloaded from some package repository. Instead for each package a bootstrap version is built from the sources using `haikuporter` and the respective build recipe from haikuports.cross. *The third-party packages are not downloaded from some package repository. Instead for each package a bootstrap version is built from the sources using <tt>haikuporter</tt> and the respective build recipe from haikuports.cross.
* `haikuporter` itself and ready-to-build ("rigged") source packages for all needed final third-party packages are copied to the image. *<tt>haikuporter</tt> itself and ready-to-build ("rigged") source packages for all needed final third-party packages are copied to the image.
Obviously the last two points are the juicy parts. Building a bootstrap third-party package -- unless it is a pure data package -- requires certain parts of Haiku; usually the headers, libroot and other libraries, and the glue code. For some Haiku libraries we do already need certain third-party packages. So there's a bit of ping pong going on: Obviously the last two points are the juicy parts. Building a bootstrap third-party package -- unless it is a pure data package -- requires certain parts of Haiku; usually the headers, libroot and other libraries, and the glue code. For some Haiku libraries we do already need certain third-party packages. So there's a bit of ping pong going on:
*Initially the build system builds a package "haiku_cross_devel_sysroot_stage1_<arch>.hpkg". It contains the essentials for cross-compiling bootstrap third-party packages, but nothing that itself depends on a third-party package. *Initially the build system builds a package "haiku_cross_devel_sysroot_stage1_<arch>.hpkg". It contains the essentials for cross-compiling bootstrap third-party packages, but nothing that itself depends on a third-party package.
*Once all third-party packages required for it have been built, a more complete "haiku_cross_devel_sysroot_<arch>.hpkg" is built. It is used to cross-compile the remaining third-party packages. *Once all third-party packages required for it have been built, a more complete "haiku_cross_devel_sysroot_<arch>.hpkg" is built. It is used to cross-compile the remaining third-party packages.
The rigged source packages (and regular source packages) are built via `haikuporter` from the regular haikuports repository checkout. haikuports contains build recipes for a lot of software. Which source packages should be built is determined by the build system by checking what packages are needed for the target build profile used by the bootstrap process. This defaults to "minimum-raw", but it can be changed by setting the jam variable "HAIKU_BOOTSTRAP_SOURCES_PROFILE" (i.e. `jam -sHAIKU_BOOTSTRAP_SOURCES_PROFILE=@release-raw -q @bootstrap-raw` will include source packages for all packages needed by release image). The rigged source packages (and regular source packages) are built via <tt>haikuporter</tt> from the regular haikuports repository checkout. haikuports contains build recipes for a lot of software. Which source packages should be built is determined by the build system by checking what packages are needed for the target build profile used by the bootstrap process. This defaults to "minimum-raw", but it can be changed by setting the jam variable "HAIKU_BOOTSTRAP_SOURCES_PROFILE" (i.e. <tt>jam -sHAIKU_BOOTSTRAP_SOURCES_PROFILE=@release-raw -q @bootstrap-raw</tt> will include source packages for all packages needed by release image).
== Haiku Architecture Ports == === Haiku Architecture Ports ===
When preparing a new Haiku architecture port for the bootstrap build the following things need to be considered: When preparing a new Haiku architecture port for the bootstrap build the following things need to be considered:
*There need to be repository definitions "build/jam/repositories/HaikuPorts/<arch>" and "build/jam/repositories/HaikuPortsCross/<arch>". The former lists the packages available for a regular Haiku, i.e. it must include at least the packages needed for a basic Haiku image that can build third-party packages. The latter lists the available bootstrap third-party packages. *There need to be repository definitions "build/jam/repositories/HaikuPorts/<arch>" and "build/jam/repositories/HaikuPortsCross/<arch>". The former lists the packages available for a regular Haiku, i.e. it must include at least the packages needed for a basic Haiku image that can build third-party packages. The latter lists the available bootstrap third-party packages.
+76 -75
View File
@@ -1,103 +1,104 @@
= Building Packages =
This page provides information regarding the package building process. The first section documents building a package with the low level command `package`. The second section refers to building packages with the `haikuporter` tool. == Building Packages ==
This page provides information regarding the package building process. The first section documents building a package with the low level command <tt>package</tt>. The second section refers to building packages with the <tt>haikuporter</tt> tool.
== Building a Package with the "package" Command == === Building a Package with the "package" Command ===
The package file format is specified in detail in a [wiki:PackageManagement/FileFormat separate document]. This section presents information from the perspective of how to build a package file with the `package` command. The package file format is specified in detail in a [[PackageManagement/FileFormat| separate document]]. This section presents information from the perspective of how to build a package file with the <tt>package</tt> command.
An hpkg file is an archive file (just like tar or zip files) that additionally contains package meta information in a separate section of the file. When building an hpkg file via the `package` command the meta information must be provided via a `.PackageInfo` file. For convenience, the file itself is added to the archive as well and can be extracted later, but it will be ignored by packagefs. An hpkg file is an archive file (just like tar or zip files) that additionally contains package meta information in a separate section of the file. When building an hpkg file via the <tt>package</tt> command the meta information must be provided via a <tt>.PackageInfo</tt> file. For convenience, the file itself is added to the archive as well and can be extracted later, but it will be ignored by packagefs.
The `.PackageInfo` file must be located in the top directory that is archived. A `package` invocation usually looks like that: The <tt>.PackageInfo</tt> file must be located in the top directory that is archived. A <tt>package</tt> invocation usually looks like that:
{{{ <pre>
package create -C foo-4.5.26-1 foo-4.5.26-1-x86.hpkg package create -C foo-4.5.26-1 foo-4.5.26-1-x86.hpkg
}}} </pre>
or (packaging a gcc2 build from within the folder): or (packaging a gcc2 build from within the folder):
{{{ <pre>
cd foo-4.5.26-1 cd foo-4.5.26-1
package create ../foo-4.5.26-1-x86_gcc2.hpkg package create ../foo-4.5.26-1-x86_gcc2.hpkg
}}} </pre>
The argument of the `-C` option specifies the directory whose contents to archive (by default the current directory), the remaining argument is the path of the package file to be built. The argument of the <tt>-C</tt> option specifies the directory whose contents to archive (by default the current directory), the remaining argument is the path of the package file to be built.
=== The .PackageInfo === ==== The .PackageInfo ====
The contents of the `.PackageInfo` adheres to a restricted driver settings syntax. It consists of name-value pairs, following this simple grammar: The contents of the <tt>.PackageInfo</tt> adheres to a restricted driver settings syntax. It consists of name-value pairs, following this simple grammar:
{{{ <pre>
package_info ::= attribute* package_info ::= attribute*
attribute ::= name value_list attribute ::= name value_list
value_list ::= value | ( "{" value* "}" ) value_list ::= value | ( "{" value* "}" )
value ::= value_item+ ( '\n' | ';' ) value ::= value_item+ ( '\n' | ';' )
}}} </pre>
`name` can be one of the attribute names defined below. `value_item` is either an unquoted string not containing any whitespace characters or a string enclosed in quotation marks (`"` or `'`) which can contain whitespace and also escaped characters (using `\`). <tt>name</tt> can be one of the attribute names defined below. <tt>value_item</tt> is either an unquoted string not containing any whitespace characters or a string enclosed in quotation marks (<tt>"</tt> or <tt>'</tt>) which can contain whitespace and also escaped characters (using <tt>\</tt>).
The supported attributes are: The supported attributes are:
* `name`: The name of the package, not including the package version. Must only contain `entity_name_char` characters. *<tt>name</tt>: The name of the package, not including the package version. Must only contain <tt>entity_name_char</tt> characters.
{{{ <pre>
entity_name_char ::= any character but '-', '/', '=', '!', '<', '>', or whitespace entity_name_char ::= any character but '-', '/', '=', '!', '<', '>', or whitespace
}}} </pre>
* `version`: The version of the package. The string must have the `version` format (see the [wiki:PackageManagement/BuildingPackages#VersionStrings Version Strings] section). *<tt>version</tt>: The version of the package. The string must have the <tt>version</tt> format (see the [[PackageManagement/BuildingPackages#VersionStrings| Version Strings]] section).
* `architecture`: The system architecture the package has been built for. Can be either of: *<tt>architecture</tt>: The system architecture the package has been built for. Can be either of:
- `any`: Any architecture (e.g. a documentation package). - <tt>any</tt>: Any architecture (e.g. a documentation package).
- `x86`: Haiku x86, built with gcc 4. - <tt>x86</tt>: Haiku x86, built with gcc 4.
- `x86_gcc2`: Haiku x86, built with gcc 2. - <tt>x86_gcc2</tt>: Haiku x86, built with gcc 2.
* `summary`: A short (one-line) description of the package. *<tt>summary</tt>: A short (one-line) description of the package.
* `description`: A longer description of the package. *<tt>description</tt>: A longer description of the package.
* `vendor`: The name of the person/organization publishing this package. *<tt>vendor</tt>: The name of the person/organization publishing this package.
* `packager`: The name and e-mail address of person that created this package (e.g. "Peter Packman <![email protected]>"). *<tt>packager</tt>: The name and e-mail address of person that created this package (e.g. "Peter Packman <[email protected]>").
* `copyrights`: A list of copyrights applying to the software contained in this package. *<tt>copyrights</tt>: A list of copyrights applying to the software contained in this package.
* `licenses`: A list of names of the licenses applying to the software contained in this package. *<tt>licenses</tt>: A list of names of the licenses applying to the software contained in this package.
* `urls`: A list of URLs referring to the packaged software's project home page. The list elements can be regular URLs or email-like named URLs (e.g. "Project Foo <!http://foo.example.com>"). *<tt>urls</tt>: A list of URLs referring to the packaged software's project home page. The list elements can be regular URLs or email-like named URLs (e.g. "Project Foo <http://foo.example.com>").
* `source-urls`: A list of URLs referring to the packaged software's source code or build instructions. Elements have the same format as those of `urls`. *<tt>source-urls</tt>: A list of URLs referring to the packaged software's source code or build instructions. Elements have the same format as those of <tt>urls</tt>.
* `flags`: A list of boolean flags applying to the package. Can contain any of the following: *<tt>flags</tt>: A list of boolean flags applying to the package. Can contain any of the following:
- `approve_license`: This package's license requires approval (i.e. must be shown to and acknowledged by user before installation). - <tt>approve_license</tt>: This package's license requires approval (i.e. must be shown to and acknowledged by user before installation).
- `system_package`: This is a system package (i.e. lives under "/boot/system") . - <tt>system_package</tt>: This is a system package (i.e. lives under "/boot/system") .
* `provides`: A list of entities provided by this package. The list elements must have the following format: *<tt>provides</tt>: A list of entities provided by this package. The list elements must have the following format:
{{{ <pre>
entity ::= entity_name [ "=" version_ref ] [ ( "compat" | "compatible" ) ">=" version_ref ] entity ::= entity_name [ "=" version_ref ] [ ( "compat" | "compatible" ) ">=" version_ref ]
entity_name ::= [ entity_type ":" ] entity_name_char+ entity_name ::= [ entity_type ":" ] entity_name_char+
entity_type ::= "lib" | "cmd" | "app" | "add_on" entity_type ::= "lib" | "cmd" | "app" | "add_on"
}}} </pre>
See the [wiki:PackageManagement/BuildingPackages#VersionStrings Version Strings] section for the `version_ref` definition. See the [[PackageManagement/BuildingPackages#VersionStrings| Version Strings]] section for the <tt>version_ref</tt> definition.
The first `version_ref` specifies the version of the provided entity. It can be omitted e.g. for abstract resolvables like "web_browser". The `version_ref` after the "compat"/"compatible" string specifies the oldest version the resolvable is backwards compatible with. The first <tt>version_ref</tt> specifies the version of the provided entity. It can be omitted e.g. for abstract resolvables like "web_browser". The <tt>version_ref</tt> after the "compat"/"compatible" string specifies the oldest version the resolvable is backwards compatible with.
The `entity_type` specifies the type of entity provided: a library ("lib"), a command line program ("cmd"), an application ("app"), or an add-on ("add-on"). The <tt>entity_type</tt> specifies the type of entity provided: a library ("lib"), a command line program ("cmd"), an application ("app"), or an add-on ("add-on").
* `requires`: A list of entities required by this package. The list elements must have the following format: *<tt>requires</tt>: A list of entities required by this package. The list elements must have the following format:
{{{ <pre>
required_entity ::= entity_name [ version_operator version_ref [ "base" ] ] required_entity ::= entity_name [ version_operator version_ref [ "base" ] ]
version_operator ::= "<" | "<=" | "==" | "!=" | ">=" | ">" version_operator ::= "<" | "<=" | "==" | "!=" | ">=" | ">"
}}} </pre>
See the [wiki:PackageManagement/BuildingPackages#VersionStrings Version Strings] section for the `version_ref` definition. If "base" is specified, the specified entity is the base package for this package. The package manager shall ensure that this package is installed in the same installation location as its base package. See the [[PackageManagement/BuildingPackages#VersionStrings| Version Strings]] section for the <tt>version_ref</tt> definition. If "base" is specified, the specified entity is the base package for this package. The package manager shall ensure that this package is installed in the same installation location as its base package.
* `supplements`: A list of entities that are supplemented by this package (i.e. this package will automatically be selected for installation if the supplemented entities are already installed). The list elements must have the `required_entity` format. *<tt>supplements</tt>: A list of entities that are supplemented by this package (i.e. this package will automatically be selected for installation if the supplemented entities are already installed). The list elements must have the <tt>required_entity</tt> format.
* `conflicts`: A list of entities that this package conflicts with (i.e. only one of both can be installed at any time). The list elements must have the `required_entity` format. *<tt>conflicts</tt>: A list of entities that this package conflicts with (i.e. only one of both can be installed at any time). The list elements must have the <tt>required_entity</tt> format.
* `freshens`: A list of entities that are being freshened by this package (i.e. this package will patch one or more files of the package(s) that provide this entity). The list elements must have the `required_entity` format. *<tt>freshens</tt>: A list of entities that are being freshened by this package (i.e. this package will patch one or more files of the package(s) that provide this entity). The list elements must have the <tt>required_entity</tt> format.
* `replaces`: A list of entities that are being replaced by this package (used if the name of a package changes, or if a package has been split). The list elements must have the `entity_name` format. *<tt>replaces</tt>: A list of entities that are being replaced by this package (used if the name of a package changes, or if a package has been split). The list elements must have the <tt>entity_name</tt> format.
* `global-writable-files`: A list of global writable file infos. The list elements must have the following format: *<tt>global-writable-files</tt>: A list of global writable file infos. The list elements must have the following format:
{{{ <pre>
global_writable_file_info ::= path [ "directory" ] [ "keep-old" | "manual" | "auto-merge" ] global_writable_file_info ::= path [ "directory" ] [ "keep-old" | "manual" | "auto-merge" ]
}}} </pre>
`path` is the relative path of the writable file or directory, starting with "settings/" or any other writable directory. If the "directory" keyword is given, the path refers to a directory. If no other keyword is given after the path respectively after the "directory" keyword, the file or directory is not included in the package. It will be created by the software or by the user. If a keyword is given, the file or directory (a default version) is included in the package and it will be extracted on package activation. The keyword specifies what shall happen when the package is updated and a previous default version of the file or directory has been modified by the user: <tt>path</tt> is the relative path of the writable file or directory, starting with "settings/" or any other writable directory. If the "directory" keyword is given, the path refers to a directory. If no other keyword is given after the path respectively after the "directory" keyword, the file or directory is not included in the package. It will be created by the software or by the user. If a keyword is given, the file or directory (a default version) is included in the package and it will be extracted on package activation. The keyword specifies what shall happen when the package is updated and a previous default version of the file or directory has been modified by the user:
- "keep-old": Indicates that the software can read old files and the user-modified file or directory should be kept. - "keep-old": Indicates that the software can read old files and the user-modified file or directory should be kept.
- "manual": Indicates that the software may not be able to read an older file and the user may have to manually adjust it. - "manual": Indicates that the software may not be able to read an older file and the user may have to manually adjust it.
- "auto-merge": Indicates that the file format is simple text and a three-way merge shall be attempted (not applicable for directories). - "auto-merge": Indicates that the file format is simple text and a three-way merge shall be attempted (not applicable for directories).
* `user-settings-files`: A list of user settings file infos. The list elements must have the following format: *<tt>user-settings-files</tt>: A list of user settings file infos. The list elements must have the following format:
{{{ <pre>
user_settings_file_info ::= path [ "directory" | "template" template_path ] user_settings_file_info ::= path [ "directory" | "template" template_path ]
}}} </pre>
`path` is the relative path of the settings file or directory, starting with "settings/". It is not included in the package. However, if `template_path` is specified, it is a path to a file included in the package that can serve as a template for the settings file. It doesn't imply any automatic action on package activation, though. If the "directory" keyword is given, the path refers to a settings directory (typical when a program creates multiple settings files). <tt>path</tt> is the relative path of the settings file or directory, starting with "settings/". It is not included in the package. However, if <tt>template_path</tt> is specified, it is a path to a file included in the package that can serve as a template for the settings file. It doesn't imply any automatic action on package activation, though. If the "directory" keyword is given, the path refers to a settings directory (typical when a program creates multiple settings files).
* `users`: A list of specifications for Unix users the packaged software requires. The list elements must have the following format: *<tt>users</tt>: A list of specifications for Unix users the packaged software requires. The list elements must have the following format:
{{{ <pre>
user: ::= name [ "real-name" real_name ] "home" home_path [ "shell" shell_path ] [ "groups" group+ ] user: ::= name [ "real-name" real_name ] "home" home_path [ "shell" shell_path ] [ "groups" group+ ]
}}} </pre>
`name` is the name of the Unix user, `real_name`, if specified, the real name of the user, `home_path` the path to the user's home directory, `shell_path` the path to the user's shell, and `group` the name of a Unix group the users is a member of. If the respective components are not specified, `name` is also used as the user's real name, "/bin/bash" is the path of the user's shell, and the user will belong to the default user group. <tt>name</tt> is the name of the Unix user, <tt>real_name</tt>, if specified, the real name of the user, <tt>home_path</tt> the path to the user's home directory, <tt>shell_path</tt> the path to the user's shell, and <tt>group</tt> the name of a Unix group the users is a member of. If the respective components are not specified, <tt>name</tt> is also used as the user's real name, "/bin/bash" is the path of the user's shell, and the user will belong to the default user group.
* `groups`: A list of names of Unix groups the packaged software requires. *<tt>groups</tt>: A list of names of Unix groups the packaged software requires.
* `post-install-scripts`: A list of paths of files included in the package, which shall be executed on package activation. Each path must start with "boot/post-install/". *<tt>post-install-scripts</tt>: A list of paths of files included in the package, which shall be executed on package activation. Each path must start with "boot/post-install/".
=== Version Strings === ==== Version Strings ====
Versions strings are used in three contexts: For the package version, for resolvable versions (`provides`), and in dependency version expressions (`requires`, `supplements`, `conflicts`, `freshens`). They are structurally identical, with the exception that the former requires a revision component (`version`), while the latter two don't (`version_ref`): Versions strings are used in three contexts: For the package version, for resolvable versions (<tt>provides</tt>), and in dependency version expressions (<tt>requires</tt>, <tt>supplements</tt>, <tt>conflicts</tt>, <tt>freshens</tt>). They are structurally identical, with the exception that the former requires a revision component (<tt>version</tt>), while the latter two don't (<tt>version_ref</tt>):
{{{ <pre>
version ::= major [ "." minor [ "." micro ] ] [ "~" pre_release ] "-" revision version ::= major [ "." minor [ "." micro ] ] [ "~" pre_release ] "-" revision
version_ref ::= major [ "." minor [ "." micro ] ] [ "~" pre_release ] [ "-" revision ] version_ref ::= major [ "." minor [ "." micro ] ] [ "~" pre_release ] [ "-" revision ]
major ::= alphanum_underline+ major ::= alphanum_underline+
@@ -105,22 +106,22 @@ minor ::= alphanum_underline+
micro ::= alphanum_underline_dot+ micro ::= alphanum_underline_dot+
pre_release ::= alphanum_underline_dot+ pre_release ::= alphanum_underline_dot+
revision ::= positive_non_zero_integer revision ::= positive_non_zero_integer
}}} </pre>
The meaning of the major, minor, and micro version parts is vendor specific. A typical, but not universal (!), convention is to increment the major version when breaking binary compatibility (i.e. version a.d.e is backwards compatible to version a.b.c for all b.c <= d.e), to increment the minor version when adding new features (in a binary compatible way), and to increment the micro version for bug fix releases. There are, however, projects that use different conventions which don't imply that e.g. version 1.4 is backwards compatible with version 1.2. Which convention is used is important for the packager to know, as it is required for a correct declaration of the compatibility versions for the provided resolvables. The compatibility version specifies the oldest version the provided resolvable is backwards compatible with, thus implying the version range requested by a dependent package the resolvable can satisfy. When following the aforementioned convention a resolvable of version 2.4.3 should have compatibility version 2 (or, semantically virtually identical, 2.0.0). Not following the convention 2.4 may be correct instead. If no compatibility version is specified, the resolvable can only satisfy dependency constraints with an exactly matching version. The meaning of the major, minor, and micro version parts is vendor specific. A typical, but not universal (!), convention is to increment the major version when breaking binary compatibility (i.e. version a.d.e is backwards compatible to version a.b.c for all b.c <= d.e), to increment the minor version when adding new features (in a binary compatible way), and to increment the micro version for bug fix releases. There are, however, projects that use different conventions which don't imply that e.g. version 1.4 is backwards compatible with version 1.2. Which convention is used is important for the packager to know, as it is required for a correct declaration of the compatibility versions for the provided resolvables. The compatibility version specifies the oldest version the provided resolvable is backwards compatible with, thus implying the version range requested by a dependent package the resolvable can satisfy. When following the aforementioned convention a resolvable of version 2.4.3 should have compatibility version 2 (or, semantically virtually identical, 2.0.0). Not following the convention 2.4 may be correct instead. If no compatibility version is specified, the resolvable can only satisfy dependency constraints with an exactly matching version.
The pre-release part of the version string has special semantics for comparison. Unlike minor and micro its presence makes the version older. E.g. version R1.0~alpha1 is considered to be older than version R1.0. When both version strings have a pre-release part, that part is compared naturally after the micro part (R1.0.1~alpha1 > R1.0 > R1.0~beta1 > R1.0~alpha2). The pre-release part of the version string has special semantics for comparison. Unlike minor and micro its presence makes the version older. E.g. version R1.0~alpha1 is considered to be older than version R1.0. When both version strings have a pre-release part, that part is compared naturally after the micro part (R1.0.1~alpha1 > R1.0 > R1.0~beta1 > R1.0~alpha2).
The revision part of the version string is assigned by the packager (not by the vendor). It allows to uniquely identify updated packages of the same vendor version of a software. The revision part of the version string is assigned by the packager (not by the vendor). It allows to uniquely identify updated packages of the same vendor version of a software.
=== Package File Names === ==== Package File Names ====
A package file name should have the following form: A package file name should have the following form:
{{{ <pre>
file_name ::= name "-" version "-" architecture ".hpkg" file_name ::= name "-" version "-" architecture ".hpkg"
}}} </pre>
=== Example package file === ==== Example package file ====
{{{ <pre>
name example name example
version 42.17-12 version 42.17-12
architecture x86_gcc2 architecture x86_gcc2
@@ -153,9 +154,9 @@ global-writable-files {
source-urls { source-urls {
"Download <http://example.com/source.zip>" "Download <http://example.com/source.zip>"
} }
}}} </pre>
== Building a Package with "haikuporter" == === Building a Package with "haikuporter" ===
`haikuporter` is a high level tool for building packages. As input it reads a build recipe file for a certain version of a software (aka port) and produces one or more packages, as declared in the recipe. A recipe specifies package requirements similar to how it is done in a `.PackageInfo` file. When asked to build a port, `haikuporter` resolves the respective dependencies and recursively builds all not-yet-built ports required for the requested port. `haikuporter` itself and a large library of recipe files are hosted at [https://github.com/haikuports/ HaikuPorts]. A detailed [https://github.com/haikuports/haikuports/wiki/HaikuPorterForPM documentation for haikuporter] and the [https://github.com/haikuports/haikuports/wiki/HaikuPorter-BuildRecipes recipe format] can also be found there. <tt>haikuporter</tt> is a high level tool for building packages. As input it reads a build recipe file for a certain version of a software (aka port) and produces one or more packages, as declared in the recipe. A recipe specifies package requirements similar to how it is done in a <tt>.PackageInfo</tt> file. When asked to build a port, <tt>haikuporter</tt> resolves the respective dependencies and recursively builds all not-yet-built ports required for the requested port. <tt>haikuporter</tt> itself and a large library of recipe files are hosted at [https://github.com/haikuports/ HaikuPorts]. A detailed [https://github.com/haikuports/haikuports/wiki/HaikuPorterForPM documentation for haikuporter] and the [https://github.com/haikuports/haikuports/wiki/HaikuPorter-BuildRecipes recipe format] can also be found there.
+13 -12
View File
@@ -1,8 +1,9 @@
= Boot Volume Directory Structure =
== Boot Volume Directory Structure ==
This is the directory layout of the boot volume: This is the directory layout of the boot volume:
{{{ <pre>
home/config home/config
<like system, but without haiku_loader, kernel_<arch>, and runtime_loader> <like system, but without haiku_loader, kernel_<arch>, and runtime_loader>
@@ -29,15 +30,15 @@ system
runtime_loader runtime_loader
trash trash
}}} </pre>
The structure mostly equals the pre-package management directory structure with the following changes: The structure mostly equals the pre-package management directory structure with the following changes:
* `common` has been removed, or more correctly it has been merged into `system`. All system-wide software is now installed (only) in `system`. *<tt>common</tt> has been removed, or more correctly it has been merged into <tt>system</tt>. All system-wide software is now installed (only) in <tt>system</tt>.
* The `develop` directory has been removed and its contents has been moved to the `system/develop` directory. *The <tt>develop</tt> directory has been removed and its contents has been moved to the <tt>system/develop</tt> directory.
* The `include` directory has been removed. Its contents lives in `develop/headers` now. *The <tt>include</tt> directory has been removed. Its contents lives in <tt>develop/headers</tt> now.
* `optional` has been removed. Optional features can just be installed via the package manager. *<tt>optional</tt> has been removed. Optional features can just be installed via the package manager.
* `share` and `etc` (in `common`) have been removed. Their contents goes to `data`, `documentation`, or `settings` (in `system` or, for packages installed there, in `home`) as appropriate. There's `settings/etc` which is where ported Unix software will usually store their global settings. *<tt>share</tt> and <tt>etc</tt> (in <tt>common</tt>) have been removed. Their contents goes to <tt>data</tt>, <tt>documentation</tt>, or <tt>settings</tt> (in <tt>system</tt> or, for packages installed there, in <tt>home</tt>) as appropriate. There's <tt>settings/etc</tt> which is where ported Unix software will usually store their global settings.
* `apps` and `preferences` have been moved to `system` for consistency. *<tt>apps</tt> and <tt>preferences</tt> have been moved to <tt>system</tt> for consistency.
* `system` and `home/config` each sport a `packages` directory, which contains the activated packages. *<tt>system</tt> and <tt>home/config</tt> each sport a <tt>packages</tt> directory, which contains the activated packages.
* `system` and `home/config` themselves are mount points for two instances of the packagefs, i.e. each contains the virtually extracted contents of the activated packages in the respective `packages` subdirectory. The directories marked with `*` are "shine-through" directories. They are not provided by the packagefs, but are the underlying directories of the boot volume. Unlike the other directories they are writable. *<tt>system</tt> and <tt>home/config</tt> themselves are mount points for two instances of the packagefs, i.e. each contains the virtually extracted contents of the activated packages in the respective <tt>packages</tt> subdirectory. The directories marked with <tt>*</tt> are "shine-through" directories. They are not provided by the packagefs, but are the underlying directories of the boot volume. Unlike the other directories they are writable.
* `system` and `home/config` each contain a directory `non-packaged` which has the same structure as their parent directory minus the shine-through directories. In the `non-packaged` directories software can be installed the traditional -- non-packaged -- way. *<tt>system</tt> and <tt>home/config</tt> each contain a directory <tt>non-packaged</tt> which has the same structure as their parent directory minus the shine-through directories. In the <tt>non-packaged</tt> directories software can be installed the traditional -- non-packaged -- way.
+252 -68
View File
@@ -1,5 +1,6 @@
[[PageOutline(2-3, Contents)]] [[PageOutline(2-3, Contents)]]
= Haiku Package File Format = == Haiku Package File Format ==
This document specifies the Haiku Package (HPKG) file format, which was designed for efficient use by Haiku's package file system. It is somewhat inspired by the [http://code.google.com/p/xar/ XAR format] (separate TOC and data heap), but aims for greater compactness (no XML for the TOC). This document specifies the Haiku Package (HPKG) file format, which was designed for efficient use by Haiku's package file system. It is somewhat inspired by the [http://code.google.com/p/xar/ XAR format] (separate TOC and data heap), but aims for greater compactness (no XML for the TOC).
@@ -10,7 +11,7 @@ Three stacked format layers can be identified:
== The Data Container Format == === The Data Container Format ===
A HPKG file consists of four sections: A HPKG file consists of four sections:
Header:: Header::
@@ -30,11 +31,11 @@ The TOC and Package Attributes sections aren't really separate sections, as they
All numbers in the HPKG are stored in big endian format or [http://en.wikipedia.org/wiki/LEB128 LEB128] encoding. All numbers in the HPKG are stored in big endian format or [http://en.wikipedia.org/wiki/LEB128 LEB128] encoding.
=== Header === ==== Header ====
The header has the following structure: The header has the following structure:
{{{ <pre>
struct hpkg_header { struct hpkg_header {
uint32 magic; uint32 magic;
uint16 header_size; uint16 header_size;
@@ -56,7 +57,7 @@ struct hpkg_header {
uint64 toc_strings_length; uint64 toc_strings_length;
uint64 toc_strings_count; uint64 toc_strings_count;
}; };
}}} </pre>
magic:: magic::
The string 'hpkg' (B_HPKG_MAGIC). The string 'hpkg' (B_HPKG_MAGIC).
@@ -101,21 +102,29 @@ struct hpkg_header {
The number of entries in the strings subsection of the TOC section. The number of entries in the strings subsection of the TOC section.
=== Heap === ==== Heap ====
The heap provides storage for arbitrary data. Data from various sources are concatenated without padding or separator, forming the uncompressed heap. A specific section of data is usually referenced (e.g. in the TOC and attributes sections) by an offset and the number of bytes. These references always point into the uncompressed heap, even if the heap is actually stored in a compressed format. The `heap_compression` field in the header specifies which format is used. The following values are defined: The heap provides storage for arbitrary data. Data from various sources are concatenated without padding or separator, forming the uncompressed heap. A specific section of data is usually referenced (e.g. in the TOC and attributes sections) by an offset and the number of bytes. These references always point into the uncompressed heap, even if the heap is actually stored in a compressed format. The <tt>heap_compression</tt> field in the header specifies which format is used. The following values are defined:
{| border=1 class="simple"
!0
!B_HPKG_COMPRESSION_NONE
!no compression
|-
| 1
| B_HPKG_COMPRESSION_ZLIB
| zlib (LZ77) compression
|}
||0||B_HPKG_COMPRESSION_NONE||no compression||
||1||B_HPKG_COMPRESSION_ZLIB||zlib (LZ77) compression||
The uncompressed heap data are divided into equally sized chunks (64 KiB). The last chunk in the heap may have a different uncompressed length from the preceding chunks. The uncompressed length of the last chunk can be derived. Each individual chunk may be stored compressed or not. The uncompressed heap data are divided into equally sized chunks (64 KiB). The last chunk in the heap may have a different uncompressed length from the preceding chunks. The uncompressed length of the last chunk can be derived. Each individual chunk may be stored compressed or not.
Unless B_HPKG_COMPRESSION_NONE is specified, a uint16 array at the end of the heap contains the actual in-file (compressed) size of each chunk (minus 1 -- 0 means 1 byte), save for the last one, which is omitted since it is implied. A chunk is only stored compressed, if compression actually saves space. That is if the chunk's compressed size equals its uncompressed size, the data aren't compressed. If B_HPKG_COMPRESSION_NONE is specified, the chunk size table is omitted entirely. Unless B_HPKG_COMPRESSION_NONE is specified, a uint16 array at the end of the heap contains the actual in-file (compressed) size of each chunk (minus 1 -- 0 means 1 byte), save for the last one, which is omitted since it is implied. A chunk is only stored compressed, if compression actually saves space. That is if the chunk's compressed size equals its uncompressed size, the data aren't compressed. If B_HPKG_COMPRESSION_NONE is specified, the chunk size table is omitted entirely.
The TOC and the package attributes sections are stored (in this order) at the end of the uncompressed heap. The offset of the package attributes section data is therefore `heap_size_uncompressed - attributes_length` and the offset of the TOC section data `heap_size_uncompressed - attributes_length - toc_length`. The TOC and the package attributes sections are stored (in this order) at the end of the uncompressed heap. The offset of the package attributes section data is therefore <tt>heap_size_uncompressed - attributes_length</tt> and the offset of the TOC section data <tt>heap_size_uncompressed - attributes_length - toc_length</tt>.
=== TOC === ==== TOC ====
The TOC section contains a list of attribute trees. An attribute has an ID, a data type, and a value, and can have child attributes. E.g.: The TOC section contains a list of attribute trees. An attribute has an ID, a data type, and a value, and can have child attributes. E.g.:
- ATTRIBUTE_ID_SHOPPING_LIST : string : "bakery" - ATTRIBUTE_ID_SHOPPING_LIST : string : "bakery"
@@ -143,11 +152,28 @@ Hence the TOC section consists of two subsections:
==== Attribute Data Types ==== ==== Attribute Data Types ====
These are the specified data type values for attributes: These are the specified data type values for attributes:
||0||B_HPKG_ATTRIBUTE_TYPE_INVALID||invalid|| {| border=1 class="simple"
||1||B_HPKG_ATTRIBUTE_TYPE_INT||signed integer|| !0
||2||B_HPKG_ATTRIBUTE_TYPE_UINT||unsigned integer|| !B_HPKG_ATTRIBUTE_TYPE_INVALID
||3||B_HPKG_ATTRIBUTE_TYPE_STRING||UTF-8 string|| !invalid
||4||B_HPKG_ATTRIBUTE_TYPE_RAW||raw data|| |-
| 1
| B_HPKG_ATTRIBUTE_TYPE_INT
| signed integer
|-
| 2
| B_HPKG_ATTRIBUTE_TYPE_UINT
| unsigned integer
|-
| 3
| B_HPKG_ATTRIBUTE_TYPE_STRING
| UTF-8 string
|-
| 4
| B_HPKG_ATTRIBUTE_TYPE_RAW
| raw data
|}
==== Strings ==== ==== Strings ====
@@ -167,7 +193,7 @@ The main TOC subsection consists of a list of attribute entries terminated by a
entries terminated by a 0 byte. entries terminated by a 0 byte.
The attribute tag encodes four pieces of information: The attribute tag encodes four pieces of information:
{{{(encoding << 11) + (hasChildren << 10) + (dataType << 7) + id + 1}}} <tt>(encoding << 11) + (hasChildren << 10) + (dataType << 7) + id + 1</tt>
encoding:: encoding::
Specifies the encoding of the attribute value as described below. Specifies the encoding of the attribute value as described below.
@@ -184,38 +210,78 @@ A value of each of the data types can be encoded in different ways, which is def
- B_HPKG_ATTRIBUTE_TYPE_INT and B_HPKG_ATTRIBUTE_TYPE_UINT: - B_HPKG_ATTRIBUTE_TYPE_INT and B_HPKG_ATTRIBUTE_TYPE_UINT:
||0||B_HPKG_ATTRIBUTE_ENCODING_INT_8_BIT||int8/uint8|| {| border=1 class="simple"
||1||B_HPKG_ATTRIBUTE_ENCODING_INT_16_BIT||int16/uint16|| !0
||2||B_HPKG_ATTRIBUTE_ENCODING_INT_32_BIT||int32/uint32|| !B_HPKG_ATTRIBUTE_ENCODING_INT_8_BIT
||3||B_HPKG_ATTRIBUTE_ENCODING_INT_64_BIT||int64/uint64|| !int8/uint8
|}
{| border=1 class="simple"
!1
!B_HPKG_ATTRIBUTE_ENCODING_INT_16_BIT
!int16/uint16
|}
{| border=1 class="simple"
!2
!B_HPKG_ATTRIBUTE_ENCODING_INT_32_BIT
!int32/uint32
|}
{| border=1 class="simple"
!3
!B_HPKG_ATTRIBUTE_ENCODING_INT_64_BIT
!int64/uint64
|}
- B_HPKG_ATTRIBUTE_TYPE_STRING: - B_HPKG_ATTRIBUTE_TYPE_STRING:
||0||B_HPKG_ATTRIBUTE_ENCODING_STRING_INLINE||null-terminated UTF-8 string|| {| border=1 class="simple"
||1||B_HPKG_ATTRIBUTE_ENCODING_STRING_TABLE||unsigned LEB128: index into string table|| !0
!B_HPKG_ATTRIBUTE_ENCODING_STRING_INLINE
!null-terminated UTF-8 string
|}
{| border=1 class="simple"
!1
!B_HPKG_ATTRIBUTE_ENCODING_STRING_TABLE
!unsigned LEB128: index into string table
|}
- B_HPKG_ATTRIBUTE_TYPE_RAW - B_HPKG_ATTRIBUTE_TYPE_RAW
||0||B_HPKG_ATTRIBUTE_ENCODING_RAW_INLINE||unsigned LEB128: size; followed by raw bytes|| {| border=1 class="simple"
||1||B_HPKG_ATTRIBUTE_ENCODING_RAW_HEAP||unsigned LEB128: size; unsigned LEB128: offset into the uncompressed heap|| !0
!B_HPKG_ATTRIBUTE_ENCODING_RAW_INLINE
!unsigned LEB128: size; followed by raw bytes
|}
{| border=1 class="simple"
!1
!B_HPKG_ATTRIBUTE_ENCODING_RAW_HEAP
!unsigned LEB128: size; unsigned LEB128: offset into the uncompressed heap
|}
=== Package Attributes ===
==== Package Attributes ====
The package attributes section contains a list of attribute trees, just like The package attributes section contains a list of attribute trees, just like
the TOC section. The structure of this section follows the TOC, i.e. there's a subsection for shared strings and a subsection that stores a list of attribute entries terminated by a 0 byte. An entry has the same format as the ones in the TOC (only using different attribute IDs). the TOC section. The structure of this section follows the TOC, i.e. there's a subsection for shared strings and a subsection that stores a list of attribute entries terminated by a 0 byte. An entry has the same format as the ones in the TOC (only using different attribute IDs).
== The Archive Format == === The Archive Format ===
This section specifies how file system objects (files, directories, symlinks) are stored in a HPKG file. It builds on top of the container format, defining the types of attributes, their order, and allowed values. This section specifies how file system objects (files, directories, symlinks) are stored in a HPKG file. It builds on top of the container format, defining the types of attributes, their order, and allowed values.
E.g. a "bin" directory, containing a symlink and a file: E.g. a "bin" directory, containing a symlink and a file:
{{{ <pre>
bin 0 2009-11-13 12:12:09 drwxr-xr-x bin 0 2009-11-13 12:12:09 drwxr-xr-x
awk 0 2009-11-13 12:11:16 lrwxrwxrwx -> gawk awk 0 2009-11-13 12:11:16 lrwxrwxrwx -> gawk
gawk 301699 2009-11-13 12:11:16 -rwxr-xr-x gawk 301699 2009-11-13 12:11:16 -rwxr-xr-x
}}} </pre>
could be represented by this attribute tree: could be represented by this attribute tree:
- B_HPKG_ATTRIBUTE_ID_DIR_ENTRY : string : "bin" - B_HPKG_ATTRIBUTE_ID_DIR_ENTRY : string : "bin"
- B_HPKG_ATTRIBUTE_ID_FILE_TYPE : uint : 1 (0x1) - B_HPKG_ATTRIBUTE_ID_FILE_TYPE : uint : 1 (0x1)
@@ -236,7 +302,7 @@ could be represented by this attribute tree:
- B_HPKG_ATTRIBUTE_ID_DATA : raw : size: 35, offset: 302379 - B_HPKG_ATTRIBUTE_ID_DATA : raw : size: 35, offset: 302379
=== Attribute IDs === ==== Attribute IDs ====
The following attribute IDs are specified by the archive format. Any other attributes will be ignored. The following attribute IDs are specified by the archive format. Any other attributes will be ignored.
@@ -262,9 +328,24 @@ The following attribute IDs are specified by the archive format. Any other attri
- '''Value:''' Type of the entry. - '''Value:''' Type of the entry.
- '''Allowed Values:''' - '''Allowed Values:'''
||0||B_HPKG_FILE_TYPE_FILE||file|| {| border=1 class="simple"
||1||B_HPKG_FILE_TYPE_DIRECTORY||directory|| !0
||2||B_HPKG_FILE_TYPE_SYMLINK||symlink|| !B_HPKG_FILE_TYPE_FILE
!file
|}
{| border=1 class="simple"
!1
!B_HPKG_FILE_TYPE_DIRECTORY
!directory
|}
{| border=1 class="simple"
!2
!B_HPKG_FILE_TYPE_SYMLINK
!symlink
|}
- '''Default Value:''' B_HPKG_FILE_TYPE_FILE - '''Default Value:''' B_HPKG_FILE_TYPE_FILE
- '''Child Attributes:''' none - '''Child Attributes:''' none
@@ -358,17 +439,17 @@ The following attribute IDs are specified by the archive format. Any other attri
- '''Child Attributes:''' none - '''Child Attributes:''' none
=== TOC Attributes === ==== TOC Attributes ====
The TOC can directly contain any number of attributes of the B_HPKG_ATTRIBUTE_ID_DIRECTORY_ENTRY type, which in turn contain descendant attributes as specified in the previous section. Any other attributes are ignored. The TOC can directly contain any number of attributes of the B_HPKG_ATTRIBUTE_ID_DIRECTORY_ENTRY type, which in turn contain descendant attributes as specified in the previous section. Any other attributes are ignored.
== The Package Format == === The Package Format ===
This section specifies how informative package attributes (package-name, version, provides, requires, ...) are stored in a HPKG file. It builds on top of the container format, defining the types of attributes, their order, and allowed values. This section specifies how informative package attributes (package-name, version, provides, requires, ...) are stored in a HPKG file. It builds on top of the container format, defining the types of attributes, their order, and allowed values.
E.g. a ".PackageInfo" file, containing a package description that is being converted into a package file: E.g. a ".PackageInfo" file, containing a package description that is being converted into a package file:
{{{ <pre>
name mypackage name mypackage
version 0.7.2-1 version 0.7.2-1
architecture x86 architecture x86
@@ -386,7 +467,7 @@ requires {
haiku >= r1 haiku >= r1
wget wget
} }
}}} </pre>
could be represented by this attribute tree: could be represented by this attribute tree:
- B_HPKG_ATTRIBUTE_ID_PACKAGE_NAME : string : "mypackage" - B_HPKG_ATTRIBUTE_ID_PACKAGE_NAME : string : "mypackage"
- B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_MAJOR : string : "0" - B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_MAJOR : string : "0"
@@ -397,7 +478,7 @@ could be represented by this attribute tree:
- B_HPKG_ATTRIBUTE_ID_PACKAGE_SUMMARY : string : "is a very nice package" - B_HPKG_ATTRIBUTE_ID_PACKAGE_SUMMARY : string : "is a very nice package"
- B_HPKG_ATTRIBUTE_ID_PACKAGE_DESCRIPTION : string : "has lots of cool features\nand is written in MyC++" - B_HPKG_ATTRIBUTE_ID_PACKAGE_DESCRIPTION : string : "has lots of cool features\nand is written in MyC++"
- B_HPKG_ATTRIBUTE_ID_PACKAGE_VENDOR : string : "Me, Myself & I, Inc." - B_HPKG_ATTRIBUTE_ID_PACKAGE_VENDOR : string : "Me, Myself & I, Inc."
- B_HPKG_ATTRIBUTE_ID_PACKAGE_PACKAGER : string : "![email protected]" - B_HPKG_ATTRIBUTE_ID_PACKAGE_PACKAGER : string : "[email protected]"
- B_HPKG_ATTRIBUTE_ID_PACKAGE_COPYRIGHT : string : "(C) 2009-2011, Me, Myself & I, Inc." - B_HPKG_ATTRIBUTE_ID_PACKAGE_COPYRIGHT : string : "(C) 2009-2011, Me, Myself & I, Inc."
- B_HPKG_ATTRIBUTE_ID_PACKAGE_LICENSE : string : "Me, Myself & I Commercial License" - B_HPKG_ATTRIBUTE_ID_PACKAGE_LICENSE : string : "Me, Myself & I Commercial License"
- B_HPKG_ATTRIBUTE_ID_PACKAGE_LICENSE : string : "MIT" - B_HPKG_ATTRIBUTE_ID_PACKAGE_LICENSE : string : "MIT"
@@ -407,11 +488,11 @@ could be represented by this attribute tree:
- B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_MINOR : string : "7" - B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_MINOR : string : "7"
- B_HPKG_ATTRIBUTE_ID_PACKAGE_REQUIRES : string : "haiku" - B_HPKG_ATTRIBUTE_ID_PACKAGE_REQUIRES : string : "haiku"
- B_HPKG_ATTRIBUTE_ID_PACKAGE_RESOLVABLE_OPERATOR : uint : 4 - B_HPKG_ATTRIBUTE_ID_PACKAGE_RESOLVABLE_OPERATOR : uint : 4
- B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_MAJOR : string : "!r1" - B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_MAJOR : string : "r1"
- B_HPKG_ATTRIBUTE_ID_PACKAGE_REQUIRES : string : "wget" - B_HPKG_ATTRIBUTE_ID_PACKAGE_REQUIRES : string : "wget"
=== Attribute IDs === ==== Attribute IDs ====
The following attribute IDs are specified by the package format. Any other attributes will be rejected. The following attribute IDs are specified by the package format. Any other attributes will be rejected.
@@ -456,8 +537,18 @@ The following attribute IDs are specified by the package format. Any other attri
- '''Value:''' Set of boolean flags applying to package. - '''Value:''' Set of boolean flags applying to package.
- '''Allowed Values:''' Any combination of the following. - '''Allowed Values:''' Any combination of the following.
||1||B_PACKAGE_FLAG_APPROVE_LICENSE||this package's license requires approval (i.e. must be shown to and acknowledged by user before installation)|| {| border=1 class="simple"
||2||B_PACKAGE_FLAG_SYSTEM_PACKAGE||this is a system package (i.e. lives under /boot/system)|| !1
!B_PACKAGE_FLAG_APPROVE_LICENSE
!this package's license requires approval (i.e. must be shown to and acknowledged by user before installation)
|}
{| border=1 class="simple"
!2
!B_PACKAGE_FLAG_SYSTEM_PACKAGE
!this is a system package (i.e. lives under /boot/system)
|}
- '''Default Value:''' 0. - '''Default Value:''' 0.
- '''Child Attributes:''' none - '''Child Attributes:''' none
@@ -466,14 +557,54 @@ The following attribute IDs are specified by the package format. Any other attri
- '''Value:''' System architecture this package was built for. - '''Value:''' System architecture this package was built for.
- '''Allowed Values:''' - '''Allowed Values:'''
||0||B_PACKAGE_ARCHITECTURE_ANY||this package doesn't depend on the system architecture|| {| border=1 class="simple"
||1||B_PACKAGE_ARCHITECTURE_X86||x86, 32-bit, built with gcc4|| !0
||2||B_PACKAGE_ARCHITECTURE_X86_GCC2||x86, 32-bit, built with gcc2|| !B_PACKAGE_ARCHITECTURE_ANY
||3||B_PACKAGE_ARCHITECTURE_SOURCE||source code, doesn't depend on the system architecture|| !this package doesn't depend on the system architecture
||4||B_PACKAGE_ARCHITECTURE_X86_64||x86-64|| |}
||5||B_PACKAGE_ARCHITECTURE_PPC||PowerPC||
||6||B_PACKAGE_ARCHITECTURE_ARM||ARM|| {| border=1 class="simple"
||7||B_PACKAGE_ARCHITECTURE_M68K||m68k|| !1
!B_PACKAGE_ARCHITECTURE_X86
!x86, 32-bit, built with gcc4
|}
{| border=1 class="simple"
!2
!B_PACKAGE_ARCHITECTURE_X86_GCC2
!x86, 32-bit, built with gcc2
|}
{| border=1 class="simple"
!3
!B_PACKAGE_ARCHITECTURE_SOURCE
!source code, doesn't depend on the system architecture
|}
{| border=1 class="simple"
!4
!B_PACKAGE_ARCHITECTURE_X86_64
!x86-64
|}
{| border=1 class="simple"
!5
!B_PACKAGE_ARCHITECTURE_PPC
!PowerPC
|}
{| border=1 class="simple"
!6
!B_PACKAGE_ARCHITECTURE_ARM
!ARM
|}
{| border=1 class="simple"
!7
!B_PACKAGE_ARCHITECTURE_M68K
!m68k
|}
- '''Child Attributes:''' none - '''Child Attributes:''' none
==== B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_MAJOR ("package:version.major") ==== ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_VERSION_MAJOR ("package:version.major") ====
@@ -525,13 +656,13 @@ The following attribute IDs are specified by the package format. Any other attri
==== B_HPKG_ATTRIBUTE_ID_PACKAGE_URL ("package:url") ==== ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_URL ("package:url") ====
- '''Type:''' string - '''Type:''' string
- '''Value:''' URL of the packaged software's project home page. - '''Value:''' URL of the packaged software's project home page.
- '''Allowed Values:''' A regular URL or an email-like named URL (e.g. "Project Foo <!http://foo.example.com>"). - '''Allowed Values:''' A regular URL or an email-like named URL (e.g. "Project Foo <http://foo.example.com>").
- '''Child Attributes:''' none - '''Child Attributes:''' none
==== B_HPKG_ATTRIBUTE_ID_PACKAGE_SOURCE_URL ("package:source-url") ==== ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_SOURCE_URL ("package:source-url") ====
- '''Type:''' string - '''Type:''' string
- '''Value:''' URL of the packaged software's source code or build instructions. - '''Value:''' URL of the packaged software's source code or build instructions.
- '''Allowed Values:''' A regular URL or an email-like named URL (e.g. "Project Foo <!http://foo.example.com>"). - '''Allowed Values:''' A regular URL or an email-like named URL (e.g. "Project Foo <http://foo.example.com>").
- '''Child Attributes:''' none - '''Child Attributes:''' none
==== B_HPKG_ATTRIBUTE_ID_PACKAGE_PROVIDES ("package:provides") ==== ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_PROVIDES ("package:provides") ====
@@ -565,12 +696,42 @@ The following attribute IDs are specified by the package format. Any other attri
- '''Value:''' Comparison operator for versions. - '''Value:''' Comparison operator for versions.
- '''Allowed Values:''' - '''Allowed Values:'''
||0||B_PACKAGE_RESOLVABLE_OP_LESS||less than the specified version|| {| border=1 class="simple"
||1||B_PACKAGE_RESOLVABLE_OP_LESS_EQUAL||less than or equal to the specified version|| !0
||2||B_PACKAGE_RESOLVABLE_OP_EQUAL||equal to the specified version|| !B_PACKAGE_RESOLVABLE_OP_LESS
||3||B_PACKAGE_RESOLVABLE_OP_NOT_EQUAL||not equal to the specified version|| !less than the specified version
||4||B_PACKAGE_RESOLVABLE_OP_GREATER_EQUAL||greater than the specified version|| |}
||5||B_PACKAGE_RESOLVABLE_OP_GREATER||greater than or equal to the specified version||
{| border=1 class="simple"
!1
!B_PACKAGE_RESOLVABLE_OP_LESS_EQUAL
!less than or equal to the specified version
|}
{| border=1 class="simple"
!2
!B_PACKAGE_RESOLVABLE_OP_EQUAL
!equal to the specified version
|}
{| border=1 class="simple"
!3
!B_PACKAGE_RESOLVABLE_OP_NOT_EQUAL
!not equal to the specified version
|}
{| border=1 class="simple"
!4
!B_PACKAGE_RESOLVABLE_OP_GREATER_EQUAL
!greater than the specified version
|}
{| border=1 class="simple"
!5
!B_PACKAGE_RESOLVABLE_OP_GREATER
!greater than or equal to the specified version
|}
- '''Child Attributes:''' none - '''Child Attributes:''' none
==== B_HPKG_ATTRIBUTE_ID_PACKAGE_SUPPLEMENTS ("package:supplements") ==== ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_SUPPLEMENTS ("package:supplements") ====
@@ -630,9 +791,24 @@ The following attribute IDs are specified by the package format. Any other attri
- '''Value:''' Specifies what to do on package update when the writable file provided by the package has been changed by the user. - '''Value:''' Specifies what to do on package update when the writable file provided by the package has been changed by the user.
- '''Allowed Values:''' - '''Allowed Values:'''
||0||B_WRITABLE_FILE_UPDATE_TYPE_KEEP_OLD||the old file shall be kept|| {| border=1 class="simple"
||1||B_WRITABLE_FILE_UPDATE_TYPE_MANUAL||the old file needs to be updated manually|| !0
||2||B_WRITABLE_FILE_UPDATE_TYPE_AUTO_MERGE||an automatic three-way merge shall be attempted|| !B_WRITABLE_FILE_UPDATE_TYPE_KEEP_OLD
!the old file shall be kept
|}
{| border=1 class="simple"
!1
!B_WRITABLE_FILE_UPDATE_TYPE_MANUAL
!the old file needs to be updated manually
|}
{| border=1 class="simple"
!2
!B_WRITABLE_FILE_UPDATE_TYPE_AUTO_MERGE
!an automatic three-way merge shall be attempted
|}
- '''Child Attributes:''' none - '''Child Attributes:''' none
==== B_HPKG_ATTRIBUTE_ID_PACKAGE_IS_WRITABLE_DIRECTORY ("package:is-writable-directory") ==== ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_IS_WRITABLE_DIRECTORY ("package:is-writable-directory") ====
@@ -640,8 +816,16 @@ The following attribute IDs are specified by the package format. Any other attri
- '''Value:''' Specifies whether the parent global writable file or user settings file attribute actually refers to a directory. - '''Value:''' Specifies whether the parent global writable file or user settings file attribute actually refers to a directory.
- '''Allowed Values:''' - '''Allowed Values:'''
||0||The parent attribute refers to a file.|| {| border=1 class="simple"
||1||The parent attribute refers to a directory.|| !0
!The parent attribute refers to a file.
|}
{| border=1 class="simple"
!1
!The parent attribute refers to a directory.
|}
- '''Child Attributes:''' none - '''Child Attributes:''' none
==== B_HPKG_ATTRIBUTE_ID_PACKAGE_SETTINGS_FILE_TEMPLATE ("package:settings-file-template") ==== ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_SETTINGS_FILE_TEMPLATE ("package:settings-file-template") ====
@@ -700,7 +884,7 @@ The following attribute IDs are specified by the package format. Any other attri
- '''Child Attributes:''' none - '''Child Attributes:''' none
== Haiku Package Repository Format == === Haiku Package Repository Format ===
Very similar to the package format, there's a Haiku Package Repository (HPKR) file format. Such a file contains informative attributes about the package repository and package attributes for all packages contained in the repository. However, this format does not contain any files. Very similar to the package format, there's a Haiku Package Repository (HPKR) file format. Such a file contains informative attributes about the package repository and package attributes for all packages contained in the repository. However, this format does not contain any files.
@@ -709,7 +893,7 @@ Two stacked format layers can be identified:
- A package format, extending the archive format with attributes for package management. - A package format, extending the archive format with attributes for package management.
=== The Data Container Format === ==== The Data Container Format ====
A HPKR file consists of three sections: A HPKR file consists of three sections:
Header:: Header::
@@ -727,7 +911,7 @@ The Repository Info and Package Attributes sections aren't really separate secti
The header has the following structure: The header has the following structure:
{{{ <pre>
struct hpkg_repo_header { struct hpkg_repo_header {
uint32 magic; uint32 magic;
uint16 header_size; uint16 header_size;
@@ -750,7 +934,7 @@ struct hpkg_repo_header {
uint64 packages_strings_length; uint64 packages_strings_length;
uint64 packages_strings_count; uint64 packages_strings_count;
}; };
}}} </pre>
magic:: magic::
The string 'hpkr' (B_HPKG_REPO_MAGIC). The string 'hpkr' (B_HPKG_REPO_MAGIC).
@@ -791,7 +975,7 @@ struct hpkg_repo_header {
The number of entries in the strings subsection of the package attributes section. The number of entries in the strings subsection of the package attributes section.
=== Attribute IDs === ==== Attribute IDs ====
The package repository format defines only the top-level attribute ID B_HPKG_ATTRIBUTE_ID_PACKAGE. An attribute with that ID represents a package. Its child attributes specify the various meta information for the package as defined in the [#AttributeIDs1 Package Format Attribute IDs] section. The package repository format defines only the top-level attribute ID B_HPKG_ATTRIBUTE_ID_PACKAGE. An attribute with that ID represents a package. Its child attributes specify the various meta information for the package as defined in the [#AttributeIDs1 Package Format Attribute IDs] section.
+6 -5
View File
@@ -1,4 +1,5 @@
= Hybrid Builds =
== Hybrid Builds ==
A hybrid build of Haiku used to be a regular Haiku built with gcc 2, also including the versions of the system libraries built with gcc 4 (or vice versa), so that programs built with either compiler could be run. Package management extends that concept and makes it more modular. Since it assigns gcc 2 and gcc 4 different (packaging) architectures ("x86_gcc2" and "x86"), we can now speak of a primary architecture -- the one the system has been built for -- and a secondary architecture -- the one the additional set of system libraries has been built for. This also fits fine with the x86_64+x86 hybrid option we might see in the future. A hybrid build of Haiku used to be a regular Haiku built with gcc 2, also including the versions of the system libraries built with gcc 4 (or vice versa), so that programs built with either compiler could be run. Package management extends that concept and makes it more modular. Since it assigns gcc 2 and gcc 4 different (packaging) architectures ("x86_gcc2" and "x86"), we can now speak of a primary architecture -- the one the system has been built for -- and a secondary architecture -- the one the additional set of system libraries has been built for. This also fits fine with the x86_64+x86 hybrid option we might see in the future.
@@ -7,17 +8,17 @@ Since the files for the secondary architecture will live in one or more separate
The following sections list the requirements for the hybrid concept and the packages built for a secondary architecture. The following sections list the requirements for the hybrid concept and the packages built for a secondary architecture.
== General Requirements == === General Requirements ===
1. Secondary architecture libraries and add-ons must live in respective "<secondary_arch>" subdirectory, where the runtime loader will only look when loading a secondary architecture executable. #Secondary architecture libraries and add-ons must live in respective "<secondary_arch>" subdirectory, where the runtime loader will only look when loading a secondary architecture executable.
2. Secondary architecture development libraries and headers must live in "<secondary_arch>" subdirectory, where only the secondary architecture compiler will look for them. 2. Secondary architecture development libraries and headers must live in "<secondary_arch>" subdirectory, where only the secondary architecture compiler will look for them.
3. Secondary architecture executables must live in a "<secondary_arch>" subdirectory, which by default isn't in PATH. The executables can be symlinked to the primary architecture "bin" directory, using a symlink name that doesn't clash with the primary architecture executable's name (by appending to the name the secondary architecture name, e.g. "grep-x86"). If there isn't a corresponding package for the primary architecture, the executables may also live directly in the "bin" directory. 3. Secondary architecture executables must live in a "<secondary_arch>" subdirectory, which by default isn't in PATH. The executables can be symlinked to the primary architecture "bin" directory, using a symlink name that doesn't clash with the primary architecture executable's name (by appending to the name the secondary architecture name, e.g. "grep-x86"). If there isn't a corresponding package for the primary architecture, the executables may also live directly in the "bin" directory.
4. Application directories should live in a "<secondary_arch>" subdirectory, unless there isn't a corresponding package for the primary architecture. 4. Application directories should live in a "<secondary_arch>" subdirectory, unless there isn't a corresponding package for the primary architecture.
== Secondary Architecture Package Requirements == === Secondary Architecture Package Requirements ===
1. A secondary architecture package must not conflict with the corresponding primary architecture package, so both can be active at the same time and in the same installation location. #A secondary architecture package must not conflict with the corresponding primary architecture package, so both can be active at the same time and in the same installation location.
2. Packages that require a command ("cmd:...") and don't need it to be for a particular architecture shouldn't need to be concerned with the primary/secondary architecture issue. E.g. "cmd:grep" should provide them with a working grep. 2. Packages that require a command ("cmd:...") and don't need it to be for a particular architecture shouldn't need to be concerned with the primary/secondary architecture issue. E.g. "cmd:grep" should provide them with a working grep.
3. Secondary architecture packages providing a command should generally provide both "cmd:<command>" and "cmd:<command>_<secondary_arch>" (e.g. "cmd:grep" and "cmd:grep_x86_gcc2"). The former makes 2. possible. The latter allows for packages to explicitly require the command for the secondary architecture (3. of the general requirements actually implies that provides item). The former should be omitted, when the behavior/output of the command is architecture dependent (e.g. in case of compilers etc.). 3. Secondary architecture packages providing a command should generally provide both "cmd:<command>" and "cmd:<command>_<secondary_arch>" (e.g. "cmd:grep" and "cmd:grep_x86_gcc2"). The former makes 2. possible. The latter allows for packages to explicitly require the command for the secondary architecture (3. of the general requirements actually implies that provides item). The former should be omitted, when the behavior/output of the command is architecture dependent (e.g. in case of compilers etc.).
4. Secondary architecture packages providing a library must provide "lib:<library>_<secondary_arch>" (e.g. "lib:libncurses_x86"). 4. Secondary architecture packages providing a library must provide "lib:<library>_<secondary_arch>" (e.g. "lib:libncurses_x86").
+19 -18
View File
@@ -1,17 +1,18 @@
= Package Management Infrastructure =
== Package Management Infrastructure ==
This pages gives an overview of what components belong to Haiku's package management infrastructure and how they work and interact. This pages gives an overview of what components belong to Haiku's package management infrastructure and how they work and interact.
* '''package''' and '''package_repo''' are command line tools for building package and package repository files. They are discussed in [wiki:PackageManagement/BuildingPackages Building Packages]. *'''package''' and '''package_repo''' are command line tools for building package and package repository files. They are discussed in [[PackageManagement/BuildingPackages| Building Packages]].
*'''packagefs''' is the file system that virtually extracts activated packages. *'''packagefs''' is the file system that virtually extracts activated packages.
*The '''package kit''' is an API for package management related programming. *The '''package kit''' is an API for package management related programming.
*The '''dependency solver''' is a part of the package kit. It solves dependencies between packages. *The '''dependency solver''' is a part of the package kit. It solves dependencies between packages.
*The '''package management daemon''' (short: package daemon) is a background process that is activated whenever the user adds packages to or removes them from one of their activation locations. It verifies that all dependencies are fulfilled (prompting the user, if necessary) and performs whatever package pre-activation/post-deactivation tasks are required. The daemon is also contacted by the package manager (after it has resolved dependencies and downloaded all required packages) to do the package de-/activation. *The '''package management daemon''' (short: package daemon) is a background process that is activated whenever the user adds packages to or removes them from one of their activation locations. It verifies that all dependencies are fulfilled (prompting the user, if necessary) and performs whatever package pre-activation/post-deactivation tasks are required. The daemon is also contacted by the package manager (after it has resolved dependencies and downloaded all required packages) to do the package de-/activation.
* The '''package manager''' provides the user interface for software installation, update, and removal. There are actually two programs, `pkgman`, a command line tool, and, `HaikuDepot`, a GUI application. *The '''package manager''' provides the user interface for software installation, update, and removal. There are actually two programs, <tt>pkgman</tt>, a command line tool, and, <tt>HaikuDepot</tt>, a GUI application.
[[Image(package-management-infrastructure.png, align=center)]] [[Image(package-management-infrastructure.png, align=center)]]
== Software Installation Locations == === Software Installation Locations ===
In Haiku there are two main locations where software is installed. "/boot/system" and "/boot/home/config". "/boot/system" is where system-wide software (i.e. software for all users), including the base system, is installed, while "/boot/home/config" is only for the user's software. The discrimination between those two doesn't make that much sense yet, but when Haiku supports multi-user it will (obviously each user will need their own home directory then, e.g. "/boot/home/<user>"). In Haiku there are two main locations where software is installed. "/boot/system" and "/boot/home/config". "/boot/system" is where system-wide software (i.e. software for all users), including the base system, is installed, while "/boot/home/config" is only for the user's software. The discrimination between those two doesn't make that much sense yet, but when Haiku supports multi-user it will (obviously each user will need their own home directory then, e.g. "/boot/home/<user>").
@@ -20,9 +21,9 @@ At both main installation locations a packagefs instance is mounted. Each instan
There are so-called shine-through directories which live on the underlying BFS volume. Normally a file system mounted at a directory would completely hide anything that is in that directory. These shine-through directories are handled specially, though; packagefs shows them just like they are on the BFS volume. One of those directories is "packages". This is necessary since otherwise it wouldn't be possible to add, remove, or update any packages. Further shine-through directories are "settings", "cache", "var", and "non-packaged". The latter is a place where software can be installed the "traditional", i.e. unpackaged, way. There are so-called shine-through directories which live on the underlying BFS volume. Normally a file system mounted at a directory would completely hide anything that is in that directory. These shine-through directories are handled specially, though; packagefs shows them just like they are on the BFS volume. One of those directories is "packages". This is necessary since otherwise it wouldn't be possible to add, remove, or update any packages. Further shine-through directories are "settings", "cache", "var", and "non-packaged". The latter is a place where software can be installed the "traditional", i.e. unpackaged, way.
== Software Installation == === Software Installation ===
=== Manual Installation === ==== Manual Installation ====
At the lowest level software can be installed by simply dropping a respective package file in a "packages" subdirectory of one of "/boot/system" or "/boot/home/config". The package daemon, a background process that sleeps most of the time, monitors the directory and, when happy with the newly added package, it asks packagefs to presents its contents on the fly in the directory structure. The package is said to be activated. Removing a package has the opposite effect. At the lowest level software can be installed by simply dropping a respective package file in a "packages" subdirectory of one of "/boot/system" or "/boot/home/config". The package daemon, a background process that sleeps most of the time, monitors the directory and, when happy with the newly added package, it asks packagefs to presents its contents on the fly in the directory structure. The package is said to be activated. Removing a package has the opposite effect.
@@ -33,34 +34,34 @@ That's why the package daemon doesn't just activate any well-formed packages. In
To avoid always having to check all dependencies when booting, the package daemon writes the last consistent state of package activations to the file "packages/administrative/activated-packages". When being mounted packagefs, reads that file and only activates the packages specified by it. If the file is missing or packages it refers to cannot be found or loaded, packagefs falls back to activating all packages in the "packages" directory. The package daemon, once started, checks the state. To avoid always having to check all dependencies when booting, the package daemon writes the last consistent state of package activations to the file "packages/administrative/activated-packages". When being mounted packagefs, reads that file and only activates the packages specified by it. If the file is missing or packages it refers to cannot be found or loaded, packagefs falls back to activating all packages in the "packages" directory. The package daemon, once started, checks the state.
=== Installation via Package Manager === ==== Installation via Package Manager ====
While manual software installation is possible, the more comfortable way is to use the package manager. The package manager has a configurable list of remote software repositories. It knows what software is available in those repositories and what is installed locally. After the user has selected software packages to be installed/deinstalled, package dependencies are resolved, and packages are downloaded and moved to their installation location. While manual software installation is possible, the more comfortable way is to use the package manager. The package manager has a configurable list of remote software repositories. It knows what software is available in those repositories and what is installed locally. After the user has selected software packages to be installed/deinstalled, package dependencies are resolved, and packages are downloaded and moved to their installation location.
The package manager prepares a transaction directory, a subdirectory in the "packages/administrative" directory, which contains the new packages. It then contacts the package management daemon (via the package kit) to perform the package activation changes. The daemon moves the new packages to the "packages" directory, moves obsolete packages to an old state directory (also a subdirectory in the "packages/administrative" directory, with the current date and time encoded in its name) and asks packagefs to activate/deactivate the respective packages. The old state directories allow recovery of old states. That is particularly interesting for the system installation location. As as safe mode/recovery option, the boot loader offers the user to select an old installation state which can then be booted into, instead of the latest state. The package manager prepares a transaction directory, a subdirectory in the "packages/administrative" directory, which contains the new packages. It then contacts the package management daemon (via the package kit) to perform the package activation changes. The daemon moves the new packages to the "packages" directory, moves obsolete packages to an old state directory (also a subdirectory in the "packages/administrative" directory, with the current date and time encoded in its name) and asks packagefs to activate/deactivate the respective packages. The old state directories allow recovery of old states. That is particularly interesting for the system installation location. As as safe mode/recovery option, the boot loader offers the user to select an old installation state which can then be booted into, instead of the latest state.
=== Application Bundles === ==== Application Bundles ====
Haiku also supports a concept that is commonly referred to as application bundles. An application bundle is a fully self-contained package that doesn't need to be installed anywhere. The implementation details have not yet been decided on. The basic idea is to either mount a dedicated packagefs with the content of such a package or have a special location where one of the three already mounted packagefs instances (likely the "/boot/home/config" one) shows that content. With a bit of Tracker (or even libbe) integration that will allow the mounted directory to be opened or the application to be started when such a package file has been double-clicked. Haiku also supports a concept that is commonly referred to as application bundles. An application bundle is a fully self-contained package that doesn't need to be installed anywhere. The implementation details have not yet been decided on. The basic idea is to either mount a dedicated packagefs with the content of such a package or have a special location where one of the three already mounted packagefs instances (likely the "/boot/home/config" one) shows that content. With a bit of Tracker (or even libbe) integration that will allow the mounted directory to be opened or the application to be started when such a package file has been double-clicked.
=== Installation Location Order and Consistency === ==== Installation Location Order and Consistency ====
Having two separate installation locations for software requires some considerations regarding their consistency and interactions. There's a well-defined order of the installation locations: "/boot/home/config", "/boot/system". This has already been the order in which on BeOS commands, libraries, and add-ons where searched (according to the environmental variables `PATH`, `LIBRARY_PATH`, and `ADDON_PATH`). That e.g. allows a user to install a new/different version of a program in "/boot/home/config" and have it override the version in "/boot/system". Having two separate installation locations for software requires some considerations regarding their consistency and interactions. There's a well-defined order of the installation locations: "/boot/home/config", "/boot/system". This has already been the order in which on BeOS commands, libraries, and add-ons where searched (according to the environmental variables <tt>PATH</tt>, <tt>LIBRARY_PATH</tt>, and <tt>ADDON_PATH</tt>). That e.g. allows a user to install a new/different version of a program in "/boot/home/config" and have it override the version in "/boot/system".
This order also needs to be the order in which package dependencies are directed. While it shall be possible to have a library in "/boot/home/config" override one in "/boot/system" and have programs installed in the latter location use the overriding library, packages in an installation location must not have dependencies that can only be resolved to packages installed in a location that is prior according to the order. E.g. a program installed in "/boot/system" must not depend on a library that is only installed in "/boot/home/config". When going multi-user that would mean the program would work for one user, but not for another one who hasn't installed the library. Consequently "/boot/system" is fully self-contained. All dependencies must be resolved within it. A safe-mode boot should be possible with only the "/boot/system" packagefs being mounted. As a consequence these constraints have to be respected when software is installed or uninstalled. This order also needs to be the order in which package dependencies are directed. While it shall be possible to have a library in "/boot/home/config" override one in "/boot/system" and have programs installed in the latter location use the overriding library, packages in an installation location must not have dependencies that can only be resolved to packages installed in a location that is prior according to the order. E.g. a program installed in "/boot/system" must not depend on a library that is only installed in "/boot/home/config". When going multi-user that would mean the program would work for one user, but not for another one who hasn't installed the library. Consequently "/boot/system" is fully self-contained. All dependencies must be resolved within it. A safe-mode boot should be possible with only the "/boot/system" packagefs being mounted. As a consequence these constraints have to be respected when software is installed or uninstalled.
Another challenge that comes with having two installation locations is that some packages have compiled-in absolute paths to their own files (e.g. data files) or to their dependencies. The former could be solved by building two different versions of a package, but that wouldn't work with the latter and would be tedious anyway. The solution are dynamically generated symbolic links in a fixed location, "/boot/system/package-links" (symlinked to "/packages"), that for each installed package and its dependencies refer to the respective installation location. Another challenge that comes with having two installation locations is that some packages have compiled-in absolute paths to their own files (e.g. data files) or to their dependencies. The former could be solved by building two different versions of a package, but that wouldn't work with the latter and would be tedious anyway. The solution are dynamically generated symbolic links in a fixed location, "/boot/system/package-links" (symlinked to "/packages"), that for each installed package and its dependencies refer to the respective installation location.
For each installed package a subdirectory named like the package (package name plus version) will be generated automatically. That subdirectory contains a symlink ".self" which refers to the installation location of the package itself as well as a symlink for each of its dependencies pointing to their installation locations. Furthermore there's a symlink ".settings" which points to the installation location's directory for global settings. E.g. for an OpenSSH package installed in "/boot/home/config" and OpenSSL installed in "/boot/system" the directory could look like this: For each installed package a subdirectory named like the package (package name plus version) will be generated automatically. That subdirectory contains a symlink ".self" which refers to the installation location of the package itself as well as a symlink for each of its dependencies pointing to their installation locations. Furthermore there's a symlink ".settings" which points to the installation location's directory for global settings. E.g. for an OpenSSH package installed in "/boot/home/config" and OpenSSL installed in "/boot/system" the directory could look like this:
{{{ <pre>
/boot/system/package-links/openssh-5.8p2-1/ /boot/system/package-links/openssh-5.8p2-1/
.self -> ../../../home/config .self -> ../../../home/config
.settings -> ../../../home/config/settings/global .settings -> ../../../home/config/settings/global
haiku -> ../.. haiku -> ../..
lib:libssl -> ../.. lib:libssl -> ../..
}}} </pre>
Installing a different, compatible version of OpenSSL in "/boot/home/config" would automatically change the respective dependency symlink. Once supporting multi-user fully, the symlinks targets would also depend on the user ID of the process that checks them, so software installed only for the user is handled correctly. Installing a different, compatible version of OpenSSL in "/boot/home/config" would automatically change the respective dependency symlink. Once supporting multi-user fully, the symlinks targets would also depend on the user ID of the process that checks them, so software installed only for the user is handled correctly.
While it depends on the location the package has been installed in where the paths refer to, the absolute paths of the package links themselves remain stable. So they can be compiled in, when a package is built, and will work regardless of where the package is installed. While it depends on the location the package has been installed in where the paths refer to, the absolute paths of the package links themselves remain stable. So they can be compiled in, when a package is built, and will work regardless of where the package is installed.
@@ -68,11 +69,11 @@ While it depends on the location the package has been installed in where the pat
Another problem the package links can solve are incompatible versions of the same package being installed in different locations. E.g. when a program and another program it depends on are installed in "/boot/system", installing an incompatible version of the latter in "/boot/home/config" will not break the former, since the dependency link will continue to point to the compatible version. With a bit of help from the runtime loader the same would work for libraries. In practice that's less of a problem, though, since libraries usually have a naming scheme and matching shared object names that already prevent mismatches. Another problem the package links can solve are incompatible versions of the same package being installed in different locations. E.g. when a program and another program it depends on are installed in "/boot/system", installing an incompatible version of the latter in "/boot/home/config" will not break the former, since the dependency link will continue to point to the compatible version. With a bit of help from the runtime loader the same would work for libraries. In practice that's less of a problem, though, since libraries usually have a naming scheme and matching shared object names that already prevent mismatches.
== Software Repositories == === Software Repositories ===
A software repository is a collection of packages, usually accessible via the internet. Haiku's package management solution allows to refer to any number of software repositories from which packages can be downloaded and installed. The structure of the respository is very simple. It's just a set of files which can be downloaded via a common protocol (HTTP or FTP). One file is the repository index file in [wiki:PackageManagement/FileFormat#HaikuPackageRepositoryFormat HPKR format]. It lists all packages that are available in the repository together with their descriptions and dependency information. It is downloaded and cached, allowing user interfaces to show the information and the dependency solver to do the computation locally. The other files are the individual package files. A software repository is a collection of packages, usually accessible via the internet. Haiku's package management solution allows to refer to any number of software repositories from which packages can be downloaded and installed. The structure of the respository is very simple. It's just a set of files which can be downloaded via a common protocol (HTTP or FTP). One file is the repository index file in [[PackageManagement/FileFormat#HaikuPackageRepositoryFormat| HPKR format]]. It lists all packages that are available in the repository together with their descriptions and dependency information. It is downloaded and cached, allowing user interfaces to show the information and the dependency solver to do the computation locally. The other files are the individual package files.
=== Standard Repositories === ==== Standard Repositories ====
There are two standard repositories for Haiku: There are two standard repositories for Haiku:
*the Haiku repository, which only contains the small set of packages that is built by Haiku's build system (haiku.hpkg, haiku_devel.hpkg, etc.) and *the Haiku repository, which only contains the small set of packages that is built by Haiku's build system (haiku.hpkg, haiku_devel.hpkg, etc.) and
@@ -86,12 +87,12 @@ For the different builds and releases there are different instances of those two
The repositories are maintained via files in the Haiku git repository. For each architecture and each repository the Haiku git repository contains a file listing the packages for that repository. For the HaikuPorts repositories the packages are listed with the respective version. For the Haiku repositories the version is implied. The repositories are maintained via files in the Haiku git repository. For each architecture and each repository the Haiku git repository contains a file listing the packages for that repository. For the HaikuPorts repositories the packages are listed with the respective version. For the Haiku repositories the version is implied.
Whenever a developer wants to update or add a HaikuPorts package, the new package file has to be uploaded to their git.haiku-os.org account and the package list file for the repository has to be adjusted accordingly. {{{jam upload-packages <packages-list>}}} can be used in order to upload the package(s) or the packages could just be {{{scp}}}'ed into the 'hpkg-upload' folder in the developer home directory on git.haiku-os.org. When that is done, the change can be pushed to git.haiku-os.org, where a push hook will analyze the change, move the new package file(s) from the developer's account to the repository directory, and build a new repository snapshot. If a package file is missing or broken, the push will be rejected with a message notifying the developer about the problem. Whenever a developer wants to update or add a HaikuPorts package, the new package file has to be uploaded to their git.haiku-os.org account and the package list file for the repository has to be adjusted accordingly. <tt>jam upload-packages <packages-list></tt> can be used in order to upload the package(s) or the packages could just be <tt>scp</tt>'ed into the 'hpkg-upload' folder in the developer home directory on git.haiku-os.org. When that is done, the change can be pushed to git.haiku-os.org, where a push hook will analyze the change, move the new package file(s) from the developer's account to the repository directory, and build a new repository snapshot. If a package file is missing or broken, the push will be rejected with a message notifying the developer about the problem.
The creation and update of repositories for official releases has to be triggered explicitly on the server. In either case the Haiku repository is built by the build service. The creation and update of repositories for official releases has to be triggered explicitly on the server. In either case the Haiku repository is built by the build service.
== The Package Kit == === The Package Kit ===
The package kit provides an API for all package management related tasks, including: The package kit provides an API for all package management related tasks, including:
*Reading and writing HPKG and HPKR files. *Reading and writing HPKG and HPKR files.
@@ -100,6 +101,6 @@ The package kit provides an API for all package management related tasks, includ
*Adding, removing, and updating packages. *Adding, removing, and updating packages.
== Localization == === Localization ===
Package files and repository index files contain text strings -- e.g. the package short and long description -- that are presented to the user. Therefore translations for these strings must be available. Since it is impractical to include the translations in the package and repository index files, they must be provided in separate files. How exactly has not been decided on yet. Package files and repository index files contain text strings -- e.g. the package short and long description -- that are presented to the user. Therefore translations for these strings must be available. Since it is impractical to include the translations in the package and repository index files, they must be provided in separate files. How exactly has not been decided on yet.
+24 -24
View File
@@ -1,52 +1,52 @@
= Migration to Package Management = == Migration to Package Management ==
This document gives an overview of what changes with the migration to package management. It has sections for different groups of Haiku users. All applying sections should be read in order. This document gives an overview of what changes with the migration to package management. It has sections for different groups of Haiku users. All applying sections should be read in order.
== Changes for Users == === Changes for Users ===
*Almost all software lives in packages and is only virtually extracted. The virtually extracted package content is read-only. *Almost all software lives in packages and is only virtually extracted. The virtually extracted package content is read-only.
* Software (i.e. packages) can be installed via the command line package manager `pkgman` -- `pkgman search/install/uninstall/update ...` searches for, installs, uninstalls, and updates packages respectively. Packages can also be installed manually by moving (not copying) them to the respective "packages" subdirectory in "/boot/system" or "/boot/home/config". *Software (i.e. packages) can be installed via the command line package manager <tt>pkgman</tt> -- <tt>pkgman search/install/uninstall/update ...</tt> searches for, installs, uninstalls, and updates packages respectively. Packages can also be installed manually by moving (not copying) them to the respective "packages" subdirectory in "/boot/system" or "/boot/home/config".
* The directory layout has changed and many directories have become read-only. Cf. [wiki:PackageManagement/DirectoryStructure DirectoryStructure] for details. *The directory layout has changed and many directories have become read-only. Cf. [[PackageManagement/DirectoryStructure| DirectoryStructure]] for details.
*The Deskbar menu works differently. It uses a new virtual directory Tracker/Deskbar feature to generate its content. Any package can contribute Deskbar menu entries by including respective symlinks in "data/deskbar/menu". The virtual directory merges the respective directories from all installation locations plus the user settings directory "/boot/home/config/settings/deskbar/menu". That means whenever a package is installed/removed its Desbar menu entries will be added/removed automatically. The user settings directory allows the user to add new entries manually. The whole behavior can be changed by overriding the default virtual directory. Before using the default virtual directory "/boot/system/data/deskbar/menu_entries" Deskbar first looks for "/boot/home/config/settings/deskbar/menu_entries". This can be a virtual directory as well, or a regular directory, or a symlink to either. E.g. making it a symlink to the "menu" directory will cause Deskbar to use only the contents of that directory, i.e. the menu contents is completely user-defined. *The Deskbar menu works differently. It uses a new virtual directory Tracker/Deskbar feature to generate its content. Any package can contribute Deskbar menu entries by including respective symlinks in "data/deskbar/menu". The virtual directory merges the respective directories from all installation locations plus the user settings directory "/boot/home/config/settings/deskbar/menu". That means whenever a package is installed/removed its Desbar menu entries will be added/removed automatically. The user settings directory allows the user to add new entries manually. The whole behavior can be changed by overriding the default virtual directory. Before using the default virtual directory "/boot/system/data/deskbar/menu_entries" Deskbar first looks for "/boot/home/config/settings/deskbar/menu_entries". This can be a virtual directory as well, or a regular directory, or a symlink to either. E.g. making it a symlink to the "menu" directory will cause Deskbar to use only the contents of that directory, i.e. the menu contents is completely user-defined.
*The MIME type management works a bit differently now as well. The database entries for the default MIME types are included in the system package and those for application MIME types are included in the package containing the respective applications. Neither of those can therefore be removed. By editing them in the FileTypes preferences application they can be overridden, though. ATM there are still a few known bugs and missing features -- e.g. application MIME types aren't automatically added/removed when installing/removing a package, and the MIME type removal functionality in FileTypes needs to be reworked. *The MIME type management works a bit differently now as well. The database entries for the default MIME types are included in the system package and those for application MIME types are included in the package containing the respective applications. Neither of those can therefore be removed. By editing them in the FileTypes preferences application they can be overridden, though. ATM there are still a few known bugs and missing features -- e.g. application MIME types aren't automatically added/removed when installing/removing a package, and the MIME type removal functionality in FileTypes needs to be reworked.
* Haiku's stage 1 boot loader (the boot block in the BFS partition) has changed. That means a Haiku partition made bootable from an old Haiku -- or more generally: with a `makebootable` that predates package management -- will not be able to boot a package management Haiku. You will have to run the new `makebootable` to make it bootable again. The new `makebootable` may or may not run on an old Haiku. The safest way is to do that from a running package management Haiku (e.g. booted off a USB stick or CD). *Haiku's stage 1 boot loader (the boot block in the BFS partition) has changed. That means a Haiku partition made bootable from an old Haiku -- or more generally: with a <tt>makebootable</tt> that predates package management -- will not be able to boot a package management Haiku. You will have to run the new <tt>makebootable</tt> to make it bootable again. The new <tt>makebootable</tt> may or may not run on an old Haiku. The safest way is to do that from a running package management Haiku (e.g. booted off a USB stick or CD).
== Changes for Application Developers == === Changes for Application Developers ===
*All development files (headers, libraries, the tool chain) have moved to "develop" in the respective installation location. Headers live in "develop/headers", development libraries in "develop/lib". Development libraries means besides static libraries also symlinks to shared libraries. The shared libraries themselves as well as all symlinks required to run a program using the library (at most one symlink per library -- the soname) live in "lib". *All development files (headers, libraries, the tool chain) have moved to "develop" in the respective installation location. Headers live in "develop/headers", development libraries in "develop/lib". Development libraries means besides static libraries also symlinks to shared libraries. The shared libraries themselves as well as all symlinks required to run a program using the library (at most one symlink per library -- the soname) live in "lib".
*Commands, libraries, add-ons, and headers for the secondary architecture of a hybrid Haiku live in an "<arch>" subdirectory of their usual location. This doesn't hold for the system headers which exist only in the primary location. *Commands, libraries, add-ons, and headers for the secondary architecture of a hybrid Haiku live in an "<arch>" subdirectory of their usual location. This doesn't hold for the system headers which exist only in the primary location.
* `setgcc` is gone. The commands of the tool chain for the secondary architecture (by default) live in "/boot/system/bin/<arch>". Prepending that path to the `PATH` environment variable would make them shadow the respective primary architecture commands -- the effect would be similar to the one `setgcc` had, but only for the current shell session. Executing the new command `setarch <arch>` will start a new shell with a respectively modified `PATH`. The commands of the secondary tool chain are also available in the standard path with a name suffixed with "-<arch>" (e.g. "gcc-x86" for the gcc 4 executable on a gcc2/gcc4 hybrid). *<tt>setgcc</tt> is gone. The commands of the tool chain for the secondary architecture (by default) live in "/boot/system/bin/<arch>". Prepending that path to the <tt>PATH</tt> environment variable would make them shadow the respective primary architecture commands -- the effect would be similar to the one <tt>setgcc</tt> had, but only for the current shell session. Executing the new command <tt>setarch <arch></tt> will start a new shell with a respectively modified <tt>PATH</tt>. The commands of the secondary tool chain are also available in the standard path with a name suffixed with "-<arch>" (e.g. "gcc-x86" for the gcc 4 executable on a gcc2/gcc4 hybrid).
* Software can be packaged using the `package` tool. Cf. [wiki:PackageManagement/BuildingPackages BuildingPackages] for more information. *Software can be packaged using the <tt>package</tt> tool. Cf. [[PackageManagement/BuildingPackages| BuildingPackages]] for more information.
* The `find_directory()` API has been partially deprecated. While there are still some use cases where it should be used, in many cases the new `find_path*()` API, respectively the `BPathFinder` class should be used instead (cf. the [http://api.haiku-os.org/FindDirectory_8h.html API documentation]). *The <tt>find_directory()</tt> API has been partially deprecated. While there are still some use cases where it should be used, in many cases the new <tt>find_path*()</tt> API, respectively the <tt>BPathFinder</tt> class should be used instead (cf. the [http://api.haiku-os.org/FindDirectory_8h.html API documentation]).
== Changes for Haiku Developers == === Changes for Haiku Developers ===
*Hybrid builds no longer require two separate generated directories. Instead the build is configured with both compilers and all output files are put in a single generated directory. *Hybrid builds no longer require two separate generated directories. Instead the build is configured with both compilers and all output files are put in a single generated directory.
*The notion of a packaging architecture has been introduced. It is mostly synonymous with the architecture, save for x86 where "x86_gcc2" refers to x86 gcc 2 and "x86" to x86 gcc 4. *The notion of a packaging architecture has been introduced. It is mostly synonymous with the architecture, save for x86 where "x86_gcc2" refers to x86 gcc 2 and "x86" to x86 gcc 4.
* Several `configure` script option have changed: *Several <tt>configure</tt> script option have changed:
* `--build-cross-tools` and `--build-cross-tools-gcc4` have been merged. The (packaging) architecture must always be specified. **<tt>--build-cross-tools</tt> and <tt>--build-cross-tools-gcc4</tt> have been merged. The (packaging) architecture must always be specified.
* `--build-cross-tools` and `--cross-tools-prefix` can be given multiple times to specify hybrid builds. Only for the first `--build-cross-tools` the path to the build tools must be given. **<tt>--build-cross-tools</tt> and <tt>--cross-tools-prefix</tt> can be given multiple times to specify hybrid builds. Only for the first <tt>--build-cross-tools</tt> the path to the build tools must be given.
For example, for building the default configuration of Haiku from a file system with proper xattr support, your configure options could look like this:[[br]] For example, for building the default configuration of Haiku from a file system with proper xattr support, your configure options could look like this:<br>
{{{$ ./configure --build-cross-tools x86_gcc2 ../buildtools --build-cross-tools x86 --use-xattr}}} <tt>$ ./configure --build-cross-tools x86_gcc2 ../buildtools --build-cross-tools x86 --use-xattr</tt>
* The new option `--target-arch` has been introduced for use on Haiku for builds with the native compiler. By default, if neither `--build-cross-tools` nor `--cross-tools-prefix` are specified the build is configured for a (hybrid) configuration matching the host system's (i.e. on a gcc2/gcc4 hybrid the build is configure for that configuration as well, on a pure gcc4 Haiku you'd get a gcc4 build). `--target-arch` overrides the default, allowing to specify the architecture to build for. The option can be given multiple times to specify a hybrid configuration. E.g. "--target-arch x86_gcc2 --target-arch x86" specifies a gcc2/gcc4 hybrid and can be used on a gcc2/gcc4 or gcc4/gcc2 Haiku. **The new option <tt>--target-arch</tt> has been introduced for use on Haiku for builds with the native compiler. By default, if neither <tt>--build-cross-tools</tt> nor <tt>--cross-tools-prefix</tt> are specified the build is configured for a (hybrid) configuration matching the host system's (i.e. on a gcc2/gcc4 hybrid the build is configure for that configuration as well, on a pure gcc4 Haiku you'd get a gcc4 build). <tt>--target-arch</tt> overrides the default, allowing to specify the architecture to build for. The option can be given multiple times to specify a hybrid configuration. E.g. "--target-arch x86_gcc2 --target-arch x86" specifies a gcc2/gcc4 hybrid and can be used on a gcc2/gcc4 or gcc4/gcc2 Haiku.
* The new option `--use-xattr-ref` can be used when extended attributes are available, but their size limit prevents use of `--use-xattr` (e.g. with ext4). The build system will use a slightly different version of the generic attribute emulation via separate files that involves tagging the attributed files with a unique ID, so there cannot be any mixups between attributes or different files when the inode ID of a file changes or files with attributes get deleted without removing their attribute files. **The new option <tt>--use-xattr-ref</tt> can be used when extended attributes are available, but their size limit prevents use of <tt>--use-xattr</tt> (e.g. with ext4). The build system will use a slightly different version of the generic attribute emulation via separate files that involves tagging the attributed files with a unique ID, so there cannot be any mixups between attributes or different files when the inode ID of a file changes or files with attributes get deleted without removing their attribute files.
* Configuring a gcc 2 build should now also work on a 64 bit system (without a 32 bit environment). Tested only on openSUSE 12.3 so far, but should also work on other Linux distributions and Unixes. The `--use-32bit` should therefore be superfluous. *Configuring a gcc 2 build should now also work on a 64 bit system (without a 32 bit environment). Tested only on openSUSE 12.3 so far, but should also work on other Linux distributions and Unixes. The <tt>--use-32bit</tt> should therefore be superfluous.
*build/jam has experienced some reorganization, particularly with respect to Haiku images and (optional) packages: *build/jam has experienced some reorganization, particularly with respect to Haiku images and (optional) packages:
* Most stuff that is built ends up in the "haiku.hpkg" and "haiku_devel.hpkg" packages (or the respective "haiku_<arch>.hpkg", "haiku_<arch>_devel.hpkg" packages for the secondary architecture). The contents of the packages is defined in the respective files in the "packages" subdirectory. **Most stuff that is built ends up in the "haiku.hpkg" and "haiku_devel.hpkg" packages (or the respective "haiku_<arch>.hpkg", "haiku_<arch>_devel.hpkg" packages for the secondary architecture). The contents of the packages is defined in the respective files in the "packages" subdirectory.
* The files defining the contents of the Haiku images live now in the "images" subdirectory. **The files defining the contents of the Haiku images live now in the "images" subdirectory.
* The "repositories" subdirectory defines external repositories. Most relevant for a regular build is the HaikuPorts repository. For each architecture there is a file defining the contents of the repository. Changes in that file require a respective version of the repository to be built. Currently that has to be done manually on the haiku-files.org server. The process will be automated soon. **The "repositories" subdirectory defines external repositories. Most relevant for a regular build is the HaikuPorts repository. For each architecture there is a file defining the contents of the repository. Changes in that file require a respective version of the repository to be built. Currently that has to be done manually on the haiku-files.org server. The process will be automated soon.
* ReleaseBuildProfiles is now DefaultBuildProfiles. **ReleaseBuildProfiles is now DefaultBuildProfiles.
*The optional packages are mostly gone. There are only a few meta optional packages left. Adding regular packages to the image is done via the AddHaikuImagePackages rule. The parameters are package names (all lower case) without the version. *The optional packages are mostly gone. There are only a few meta optional packages left. Adding regular packages to the image is done via the AddHaikuImagePackages rule. The parameters are package names (all lower case) without the version.
*All build variables that depend on the architecture and aren't only relevant to the primary architecture have been renamed to have a "_<arch>" suffix (e.g. TARGET_GCC_<arch>, TARGET_DEFINES_<arch>, etc.). The variables are mostly only used by rule implementations, so this has not that much of an impact on Jamfiles. *All build variables that depend on the architecture and aren't only relevant to the primary architecture have been renamed to have a "_<arch>" suffix (e.g. TARGET_GCC_<arch>, TARGET_DEFINES_<arch>, etc.). The variables are mostly only used by rule implementations, so this has not that much of an impact on Jamfiles.
*There are new build variables HAIKU_PACKAGING_ARCHS and TARGET_PACKAGING_ARCH[S]. The plural versions are set to the list of all configured architectures, e.g. for a gcc2/gcc4 hybrid "x86_gcc2 x86". TARGET_PACKAGING_ARCH is set to the current architecture. Usually that means the primary architecture. In some cases (mostly for libraries) a target has to be built for all architectures. That is done in a loop which sets TARGET_PACKAGING_ARCH (and other variables) according to the architecture handled in that iteration. Cf. [https://github.com/haiku/HaikuPM/blob/package-management/src/kits/textencoding/Jamfile src/kits/textencoding/Jamfile] for a small example. *There are new build variables HAIKU_PACKAGING_ARCHS and TARGET_PACKAGING_ARCH[S]. The plural versions are set to the list of all configured architectures, e.g. for a gcc2/gcc4 hybrid "x86_gcc2 x86". TARGET_PACKAGING_ARCH is set to the current architecture. Usually that means the primary architecture. In some cases (mostly for libraries) a target has to be built for all architectures. That is done in a loop which sets TARGET_PACKAGING_ARCH (and other variables) according to the architecture handled in that iteration. Cf. [https://github.com/haiku/HaikuPM/blob/package-management/src/kits/textencoding/Jamfile src/kits/textencoding/Jamfile] for a small example.
*Build features (as defined in "build/jam/BuildFeatures") work differently now. Instead of build variables there are dedicated rules to deal with build features (FIsBuildFeatureEnabled, UseBuildFeatureHeaders, BuildFeatureAttribute). Cf. [https://github.com/haiku/HaikuPM/blob/package-management/src/add-ons/mail_daemon/inbound_protocols/pop3/Jamfile src/add-ons/mail_daemon/inbound_protocols/pop3/Jamfile] for an example. *Build features (as defined in "build/jam/BuildFeatures") work differently now. Instead of build variables there are dedicated rules to deal with build features (FIsBuildFeatureEnabled, UseBuildFeatureHeaders, BuildFeatureAttribute). Cf. [https://github.com/haiku/HaikuPM/blob/package-management/src/add-ons/mail_daemon/inbound_protocols/pop3/Jamfile src/add-ons/mail_daemon/inbound_protocols/pop3/Jamfile] for an example.
* The semantics of the "update" build profile action has changed somewhat, since due to the packages we now have two container levels, the image and the package. A `jam -q @alpha-raw update libbe.so` will first update libbe.so in the haiku.hpkg package and then update haiku.hpkg in the image. A `jam -q @alpha-raw update haiku.hpkg` will update "haiku.hpkg" in the image, but "haiku.hpkg" will not be rebuilt. If that is desired, it first has to be rebuilt explicitly -- via `jam -q haiku.hpkg`. Note that this might be problematic as well, since which optional build features are active depends on the specified build profile. *The semantics of the "update" build profile action has changed somewhat, since due to the packages we now have two container levels, the image and the package. A <tt>jam -q @alpha-raw update libbe.so</tt> will first update libbe.so in the haiku.hpkg package and then update haiku.hpkg in the image. A <tt>jam -q @alpha-raw update haiku.hpkg</tt> will update "haiku.hpkg" in the image, but "haiku.hpkg" will not be rebuilt. If that is desired, it first has to be rebuilt explicitly -- via <tt>jam -q haiku.hpkg</tt>. Note that this might be problematic as well, since which optional build features are active depends on the specified build profile.
*There's a new build profile action "update-packages". It updates all packages, empties "/boot/system/packages" in the image, and copies the updated packages there. It's a poor man's system update. Packages you have installed manually will be removed. The old "update-all" build profile action still exits. It has the effect of "update-packages" and additionally replaces all other files that are usually copied to the image. *There's a new build profile action "update-packages". It updates all packages, empties "/boot/system/packages" in the image, and copies the updated packages there. It's a poor man's system update. Packages you have installed manually will be removed. The old "update-all" build profile action still exits. It has the effect of "update-packages" and additionally replaces all other files that are usually copied to the image.
== Changes for Porters == === Changes for Porters ===
*The format of the recipe (formerly bep) files has changed. Many recipes have not been updated yet. haikuporter also has changed significantly. Cf. the [https://github.com/haikuports/haikuports/wiki/HaikuPorterForPM haikuporter documentation] for more information. *The format of the recipe (formerly bep) files has changed. Many recipes have not been updated yet. haikuporter also has changed significantly. Cf. the [https://github.com/haikuports/haikuports/wiki/HaikuPorterForPM haikuporter documentation] for more information.
+64 -65
View File
@@ -1,22 +1,21 @@
[[PageOutline(2-3, Contents)]] == Package Management Ideas ==
= Package Management Ideas =
This page is a place to hash out ideas regarding Haiku's package management (and creation). The following is a draft specification for the package management system to be included in R1. It is based on (1), (2) and the discussion in (5). The draft does not yet cover everything from (1), however. This page is a place to hash out ideas regarding Haiku's package management (and creation). The following is a draft specification for the package management system to be included in R1. It is based on (1), (2) and the discussion in (5). The draft does not yet cover everything from (1), however.
== Requirements == === Requirements ===
This section describes the intended user experience. This section describes the intended user experience.
=== HaikuBits === ==== HaikuBits ====
HaikuBits is a complete directory of software for the Haiku platform. It is the one place a user needs when looking for Haiku software. While it does not host all software binaries, it does list 99.9% of all software available for Haiku. HaikuBits is a complete directory of software for the Haiku platform. It is the one place a user needs when looking for Haiku software. While it does not host all software binaries, it does list 99.9% of all software available for Haiku.
Software is classified into a number of categories. For each software, HaikuBits provides a short description, one or more screenshots, a link to the author's homepage and a download link. Additionally, a software's page has a community rating and important information such as security notices. Software is classified into a number of categories. For each software, HaikuBits provides a short description, one or more screenshots, a link to the author's homepage and a download link. Additionally, a software's page has a community rating and important information such as security notices.
=== Bundles === ==== Bundles ====
A [http://en.wikipedia.org/wiki/Application_Bundle bundle] is a single file that contains the binaries, data files and documentation. This makes software self-contained and easy to handle. An application can be run by double-clicking the bundle icon. Obvious exceptions to this rule are drivers and libraries. These have to be installed to be of any use. The contents of a bundle can be inspected by opening the bundle by means of a context-menu option which opens the bundle in Tracker just like a directory. A [http://en.wikipedia.org/wiki/Application_Bundle bundle] is a single file that contains the binaries, data files and documentation. This makes software self-contained and easy to handle. An application can be run by double-clicking the bundle icon. Obvious exceptions to this rule are drivers and libraries. These have to be installed to be of any use. The contents of a bundle can be inspected by opening the bundle by means of a context-menu option which opens the bundle in Tracker just like a directory.
Optionally, an '''application''' bundle can be '''installed''' by moving it to {{{/boot/apps}}} (system-wide) or in {{{/boot/home/<user>/apps}}} (user-local). Another option is to right-click the icon and select "install for everyone" (only admins) or "install only for me". Any initial configuration (accepting a license) can be performed the first time a bundle is being run. Optionally, an '''application''' bundle can be '''installed''' by moving it to <tt>/boot/apps</tt> (system-wide) or in <tt>/boot/home/<user>/apps</tt> (user-local). Another option is to right-click the icon and select "install for everyone" (only admins) or "install only for me". Any initial configuration (accepting a license) can be performed the first time a bundle is being run.
'''Libaries''' can be installed the same way. The user normally does not have to install libraries manually, as the package manager will do so when it is needed, asking the user for permission. '''Libaries''' can be installed the same way. The user normally does not have to install libraries manually, as the package manager will do so when it is needed, asking the user for permission.
@@ -24,62 +23,62 @@ When an application bundle has been installed, shortcuts to the application appe
'''Drivers''' ... ? '''Drivers''' ... ?
{{{ <pre>
#!div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;" #div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;"
'''brecht''': I don't like Waldemar's idea of having bundles spread all around the filesystem, hence the clear distinction between installed and non-installed bundles. This might be a necessity for multi-user too. '''brecht''': I don't like Waldemar's idea of having bundles spread all around the filesystem, hence the clear distinction between installed and non-installed bundles. This might be a necessity for multi-user too.
'''axeld''': I have to agree with brecht. And also, I like the package file system best, as it also solves on how to deal with ported software, and libraries as well. Only drivers would probably need special treatment (depends on how early the package file system is available, but I guess that could be made work as well, like having a "actually install on disk" option for driver packages). '''axeld''': I have to agree with brecht. And also, I like the package file system best, as it also solves on how to deal with ported software, and libraries as well. Only drivers would probably need special treatment (depends on how early the package file system is available, but I guess that could be made work as well, like having a "actually install on disk" option for driver packages).
'''wkornewald''': My original intention was to not force the user through any installation procedure. Simply download the app and run it directly from the downloads folder to see if it works well and if yes the user can move it to the "Apps" folder. '''wkornewald''': My original intention was to not force the user through any installation procedure. Simply download the app and run it directly from the downloads folder to see if it works well and if yes the user can move it to the "Apps" folder.
}}} </pre>
=== Management === ==== Management ====
'''Uninstallation''' is performed by simply removing the bundle from {{{/boot/apps}}} or {{{/boot/home/<user>/apps}}}. '''Uninstallation''' is performed by simply removing the bundle from <tt>/boot/apps</tt> or <tt>/boot/home/<user>/apps</tt>.
A user's application '''settings''' are kept when a bundle is uninstalled or deleted. The system however provides a comprehensive listing of applications for which user settings exist. The user can choose to delete settings for each of the bundles. A user's application '''settings''' are kept when a bundle is uninstalled or deleted. The system however provides a comprehensive listing of applications for which user settings exist. The user can choose to delete settings for each of the bundles.
Application folders ({{{/boot/apps}}} and {{{/boot/home/<user>/apps}}}) display the list of installed bundles including information (description, availability of updates, security risk warning), just like the mail folder in BeOS displays emails. Application folders (<tt>/boot/apps</tt> and <tt>/boot/home/<user>/apps</tt>) display the list of installed bundles including information (description, availability of updates, security risk warning), just like the mail folder in BeOS displays emails.
{{{ <pre>
#!div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;" #div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;"
'''wkornewald''': The system should automatically remove settings of deleted apps after a certain amount of time. You don't really want the user to manually clean up his system. The cleanup delay should be long enough to allow the user to update an app by deleting the old version and downloading the new version and it should be long enough to allow the user to "undo" his action. Maybe 1-2 months is fine. '''wkornewald''': The system should automatically remove settings of deleted apps after a certain amount of time. You don't really want the user to manually clean up his system. The cleanup delay should be long enough to allow the user to update an app by deleting the old version and downloading the new version and it should be long enough to allow the user to "undo" his action. Maybe 1-2 months is fine.
'''pulkomandy''': Deleting files without asking doesn't look right to me. Either delete them as soon as the app is removed, or keep them forever. I like Debian/apt way of doing it : when you delete a package, it's listed in the package manager as 'removed, residual config files' and you can remove these from there. '''pulkomandy''': Deleting files without asking doesn't look right to me. Either delete them as soon as the app is removed, or keep them forever. I like Debian/apt way of doing it : when you delete a package, it's listed in the package manager as 'removed, residual config files' and you can remove these from there.
}}} </pre>
=== Dependencies === ==== Dependencies ====
The user does not have to worry about dependencies. If a bundle depends on other bundles, the package manager should make sure they are fulfilled (on run). If a dependency is not fulfilled, the system will ask the user for confirmation and automatically install any required bundles. This requires an internet connection. The user does not have to worry about dependencies. If a bundle depends on other bundles, the package manager should make sure they are fulfilled (on run). If a dependency is not fulfilled, the system will ask the user for confirmation and automatically install any required bundles. This requires an internet connection.
Alternatively, a user can download a '''fat bundle''' that includes the dependencies as well. Alternatively, a user can download a '''fat bundle''' that includes the dependencies as well.
=== Updates === ==== Updates ====
All ''installed software'' can be checked for new versions. For each of the installed applications, the changes with respect to the installed version can be displayed. Updates are flagged as 'strongly recommended' when security problems are known. The user can select which applications he/she would like to update and have the system perform the updates. All ''installed software'' can be checked for new versions. For each of the installed applications, the changes with respect to the installed version can be displayed. Updates are flagged as 'strongly recommended' when security problems are known. The user can select which applications he/she would like to update and have the system perform the updates.
By default, the updater tool does not show libraries in order to keep the list of updates as short as possible and understandable by the non-technical user. Libaries should only be updated when there are known problems with them. By default, the updater tool does not show libraries in order to keep the list of updates as short as possible and understandable by the non-technical user. Libaries should only be updated when there are known problems with them.
== Implementation == === Implementation ===
In this section the implementation of the system is discussed. In this section the implementation of the system is discussed.
=== Bundles === ==== Bundles ====
A bundle is a compressed disk image that contains: A bundle is a compressed disk image that contains:
*application executable(s) & data *application executable(s) & data
*metadata *metadata
* name ***name
* version ***version
* (revision?) ***(revision?)
* hash (integrity-check) ***hash (integrity-check)
* author ***author
* homepage ***homepage
* license ***license
* category (for grouping applications in the Deskbar menu) ***category (for grouping applications in the Deskbar menu)
*shortcuts to appear in the Deskbar menu *shortcuts to appear in the Deskbar menu
* a default shortcut to run when the bundle is being "run" ***a default shortcut to run when the bundle is being "run"
* right-clicking bundle could offer the option of opening a help document about the application ***right-clicking bundle could offer the option of opening a help document about the application
There are a number of different bundle types: There are a number of different bundle types:
*application *application
@@ -90,7 +89,7 @@ There are a number of different bundle types:
Library, driver and font bundles have to be installed. Library, driver and font bundles have to be installed.
=== Install or not? === ==== Install or not? ====
As bundles have to be compressed for distribution, they will need to be uncompressed at some point. There are two options: As bundles have to be compressed for distribution, they will need to be uncompressed at some point. There are two options:
*Before using a bundle, it is decompressed. This is very similar to installing, which many wanted to avoid. *Before using a bundle, it is decompressed. This is very similar to installing, which many wanted to avoid.
@@ -98,34 +97,34 @@ As bundles have to be compressed for distribution, they will need to be uncompre
It is possible to split application bundles into two sub-types; those that have to be installed, and those that can be run as-is. It is possible to split application bundles into two sub-types; those that have to be installed, and those that can be run as-is.
{{{ <pre>
#!div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;" #div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;"
'''brecht''': While I initially liked the idea of not having to install software, I now feel that it is not suited for all types of applications and games. While we can make a differentiate between 'large' and 'small' applications and require installation of large apps (or suffer from poor performance), this feels like a bit of a kludge. Is is really that bad to 'install' software once? Installing can be reduced to decompressing and should not bother the user much. If the user knows he has to install '''all''' bundles, there can be no confusion. '''brecht''': While I initially liked the idea of not having to install software, I now feel that it is not suited for all types of applications and games. While we can make a differentiate between 'large' and 'small' applications and require installation of large apps (or suffer from poor performance), this feels like a bit of a kludge. Is is really that bad to 'install' software once? Installing can be reduced to decompressing and should not bother the user much. If the user knows he has to install '''all''' bundles, there can be no confusion.
'''wkornewald''': I don't fully remember my original proposal (it's been several years :), but I think there's a middle-path: When the bundle is opened for the first time it's decompressed and cached automatically. When the bundle is deleted from the file system the cache is cleaned, too. That way you have the best of both worlds and the user only has a slow first start, but all subsequent app starts will be fast. Maybe the cache itself could be a single uncompressed bundle/image file if that's more efficient than having lots of small files spread over the main file system. A large coherent file can probably be read into memory much faster and should speed up app starts noticeably. '''wkornewald''': I don't fully remember my original proposal (it's been several years :), but I think there's a middle-path: When the bundle is opened for the first time it's decompressed and cached automatically. When the bundle is deleted from the file system the cache is cleaned, too. That way you have the best of both worlds and the user only has a slow first start, but all subsequent app starts will be fast. Maybe the cache itself could be a single uncompressed bundle/image file if that's more efficient than having lots of small files spread over the main file system. A large coherent file can probably be read into memory much faster and should speed up app starts noticeably.
}}} </pre>
=== File System Organization === ==== File System Organization ====
Native Haiku applications are contained in their own directory as before. Organization of ports is more complicated. Native Haiku applications are contained in their own directory as before. Organization of ports is more complicated.
=== Merged === ==== Merged ====
By means of a union pkgfs (3). All ports are mounted under {{{/boot/common}}}. By means of a union pkgfs (3). All ports are mounted under <tt>/boot/common</tt>.
It is not clear how multiple versions of libraries and applications can be handled in this scheme. It is not clear how multiple versions of libraries and applications can be handled in this scheme.
=== Self-contained === ==== Self-contained ====
By means of assignfs (4). Each port receives its own unique assign: {{{/boot/apps/<port>-<version>-<revision>}}} By means of assignfs (4). Each port receives its own unique assign: <tt>/boot/apps/<port>-<version>-<revision></tt>
{{{ <pre>
#!div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;" #div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;"
'''axeld''': I think the best solution would be a unionfs approach: the package file system would just blend in the packages where needed. User packages would be merged with the contents of config/, while system wide ones would be merged with the contents of /boot/common/. '''axeld''': I think the best solution would be a unionfs approach: the package file system would just blend in the packages where needed. User packages would be merged with the contents of config/, while system wide ones would be merged with the contents of /boot/common/.
}}} </pre>
=== Settings === ==== Settings ====
global settings/user settings global settings/user settings
@@ -133,7 +132,7 @@ global settings/user settings
how to handle how to handle
=== Dependency Hell === ==== Dependency Hell ====
[http://en.wikipedia.org/wiki/Dependency_hell Dependency hell] is a problem mostly for ports. That does not mean it can be ignored. At least in the early years of Haiku, ports will be an important source of software. [http://en.wikipedia.org/wiki/Dependency_hell Dependency hell] is a problem mostly for ports. That does not mean it can be ignored. At least in the early years of Haiku, ports will be an important source of software.
@@ -163,23 +162,23 @@ Having an application use a particular library version can be done by manipulati
Note the important difference with typical Linux package management systems. In Linux, the repository typically offers only one version of a particular package. This is the result of keeping all packages in the repository in sync, in order to avoid conflicting dependencies. In the proposed system however, the user is free to install any version of a bundle, as there is no need for any global synchronization of all bundles. Note the important difference with typical Linux package management systems. In Linux, the repository typically offers only one version of a particular package. This is the result of keeping all packages in the repository in sync, in order to avoid conflicting dependencies. In the proposed system however, the user is free to install any version of a bundle, as there is no need for any global synchronization of all bundles.
{{{ <pre>
#!div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;" #div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;"
'''axeld''': while having a central repository is a good thing, I don't think our package manager should be based on that idea. I would allow each package to define its own sources (the user can prevent that, of course). That way, we avoid the situation of having to choose between outdated repositories, and unstable software (or even having to build it on your own) like you usually have to do in a Linux distribution. The central repository should also be a fallback, though, and try to host most library packages.[[BR]] '''axeld''': while having a central repository is a good thing, I don't think our package manager should be based on that idea. I would allow each package to define its own sources (the user can prevent that, of course). That way, we avoid the situation of having to choose between outdated repositories, and unstable software (or even having to build it on your own) like you usually have to do in a Linux distribution. The central repository should also be a fallback, though, and try to host most library packages.<br>
Since we do care about binary compatibility, and stable APIs, having a central repository is not necessary, or something desirable at all IMO. Since we do care about binary compatibility, and stable APIs, having a central repository is not necessary, or something desirable at all IMO.
'''brecht''': I agree. I see the repository more as a central entity keeping track of all software versions and the dependencies between them. This dependency information is updated based on user feedback. I don't think it is necessary to have the repository be the one and only source for bundles, however. It can keep instead a list of available mirrors. However, it is probably a good idea to have one large reliable mirror (hosting the most important bundles) managed by Haiku Inc. alongside the repository in order not to be too dependent on third parties. '''brecht''': I agree. I see the repository more as a central entity keeping track of all software versions and the dependencies between them. This dependency information is updated based on user feedback. I don't think it is necessary to have the repository be the one and only source for bundles, however. It can keep instead a list of available mirrors. However, it is probably a good idea to have one large reliable mirror (hosting the most important bundles) managed by Haiku Inc. alongside the repository in order not to be too dependent on third parties.
}}} </pre>
=== libalpm === ==== libalpm ====
(and it's tool: pacman) (and it's tool: pacman)
libalpm is the package management library used on ArchLinux, most people know it as "pacman" since that's the main tool to use, however, all the functionality is part of the libalpm library which could be utilized to create a nice GUI frontend for the package manager. libalpm is the package management library used on ArchLinux, most people know it as "pacman" since that's the main tool to use, however, all the functionality is part of the libalpm library which could be utilized to create a nice GUI frontend for the package manager.
It of course can also be adapted. It of course can also be adapted.
It uses libarchive to extract archives, and either libdownload or libfetch to download files - although one can also have it use an external command, like curl or wget.[[BR]] It uses libarchive to extract archives, and either libdownload or libfetch to download files - although one can also have it use an external command, like curl or wget.<br>
The current status is this:[[BR]] The current status is this:<br>
As far as libfetch is concerned: compiles and is linked to, but it doesn't really work, so I'm using curl instead - it works like a charm.[[BR]] As far as libfetch is concerned: compiles and is linked to, but it doesn't really work, so I'm using curl instead - it works like a charm.<br>
The important part: libarchive needed some work to support zip files in a useful way. Basically, it now supports seeking (which it didn't before), the central-directory headers for ZIP files (so it supports stuff like symlinks), and BeOS file attributes! Also, when reading from a source which doesn't allow seeking (... which are... none - on our case) it simply reads the local headers, but can also - if explicitly requested - provide "update"-entries to update the raw data when the central directory is reached (but those are of no importance anymore). The important part: libarchive needed some work to support zip files in a useful way. Basically, it now supports seeking (which it didn't before), the central-directory headers for ZIP files (so it supports stuff like symlinks), and BeOS file attributes! Also, when reading from a source which doesn't allow seeking (... which are... none - on our case) it simply reads the local headers, but can also - if explicitly requested - provide "update"-entries to update the raw data when the central directory is reached (but those are of no importance anymore).
What's good about libalpm? Well, it provides useful configuring mechanisms, it stores dependencies and can also give you a list of which packages require a certain package. It keeps a database containing package information, including a file list. Configuration files in packages can be listed as such, which causes them to be installed as *.pacnew when they are upgraded (unless the new and old files equal - an md5 sum check is used there.) What's good about libalpm? Well, it provides useful configuring mechanisms, it stores dependencies and can also give you a list of which packages require a certain package. It keeps a database containing package information, including a file list. Configuration files in packages can be listed as such, which causes them to be installed as *.pacnew when they are upgraded (unless the new and old files equal - an md5 sum check is used there.)
@@ -195,7 +194,7 @@ It allows you to find the owning package of a file in the filesystem as well as
Where does it get the packages form? Where does it get the packages form?
Two possibilities: One can use package files directly - which could be made in such a way that you could also just unzip them. In fact, it might be useful to put the .PKGINFO into the zip file as some extra data which is not unzipped when simply using `unzip´, although package creation is easier if it's just a file. Two possibilities: One can use package files directly - which could be made in such a way that you could also just unzip them. In fact, it might be useful to put the .PKGINFO into the zip file as some extra data which is not unzipped when simply using `unzip´, although package creation is easier if it's just a file.
The other one being repositories. The pacman utility currently allows you to list repositories like this in pacman.conf: The other one being repositories. The pacman utility currently allows you to list repositories like this in pacman.conf:
{{{ <pre>
[core] [core]
Server = file:///MyRepositories Server = file:///MyRepositories
@@ -204,10 +203,10 @@ Server = file:///MyRepositories
[public] [public]
Server = http://www.public-repository.com/ Server = http://www.public-repository.com/
}}} </pre>
When you synchronize the repository databases, pacman downloads the file <Server>/<Reponame>.db.tar.gz which contains a list of packages with dependencies. When you synchronize the repository databases, pacman downloads the file <Server>/<Reponame>.db.tar.gz which contains a list of packages with dependencies.
When you install a package from such a repository, it downloads them from the very same location: <Server>/<Package File> When you install a package from such a repository, it downloads them from the very same location: <Server>/<Package File>
The repo.db.tar.gz files are currently created using the tools `repo-add` and `repo-remove` provided in the pacman package. Those extract information from the .PKGINFO file and put it into the database which can be used as a repository then. The repo.db.tar.gz files are currently created using the tools <tt>repo-add</tt> and <tt>repo-remove</tt> provided in the pacman package. Those extract information from the .PKGINFO file and put it into the database which can be used as a repository then.
Here's a little log of using pacman to install a package file, and bundle a package with dependencies together into one package file. Here's a little log of using pacman to install a package file, and bundle a package with dependencies together into one package file.
http://stud4.tuwien.ac.at/~e0725517/using-pacman-on-haiku.log.txt http://stud4.tuwien.ac.at/~e0725517/using-pacman-on-haiku.log.txt
@@ -221,15 +220,15 @@ http://stud4.tuwien.ac.at/~e0725517/using-pacman-on-haiku.log.txt
==== Cons ==== ==== Cons ====
*Likely to cause unwanted restrictions in the package management system. *Likely to cause unwanted restrictions in the package management system.
{{{ <pre>
#!div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;" #div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;"
'''brecht''': I don't have a detailed view of pacman yet, but as far as I can see it is very similar to your average Linux package manager. As we want software management to be fundamentally different from the way Linux distributions handle it (because it simply is not a very elegant solution), I don't think pacman is a good choice. '''brecht''': I don't have a detailed view of pacman yet, but as far as I can see it is very similar to your average Linux package manager. As we want software management to be fundamentally different from the way Linux distributions handle it (because it simply is not a very elegant solution), I don't think pacman is a good choice.
'''Blub''': Let me clarify: I was not suggesting to use 'pacman' as a package manager, I was just thinking that its library could be a useful codebase for a package-database, to keep track of dependencies, available packages and updates. It 'could' be used to unpack/install packages into a 'specified' folder, like /boot/apps (and even install dependencies into the same folder if wanted), or, it could just as well be used to simply keep track of where which package has been installed to without worrying about the actual contents.[[BR]] '''Blub''': Let me clarify: I was not suggesting to use 'pacman' as a package manager, I was just thinking that its library could be a useful codebase for a package-database, to keep track of dependencies, available packages and updates. It 'could' be used to unpack/install packages into a 'specified' folder, like /boot/apps (and even install dependencies into the same folder if wanted), or, it could just as well be used to simply keep track of where which package has been installed to without worrying about the actual contents.<br>
Although when stripping the code to unpack the archives and keeping track of their files it is indeed better to create something new.[[BR]] Although when stripping the code to unpack the archives and keeping track of their files it is indeed better to create something new.<br>
}}} </pre>
== Brainstorming Results from BeGeistert == === Brainstorming Results from BeGeistert ===
These features were discussed/written down at BeGeistert: These features were discussed/written down at BeGeistert:
@@ -249,7 +248,7 @@ These features were discussed/written down at BeGeistert:
*Quality of packages (QA integrated into the process of releasing a package) *Quality of packages (QA integrated into the process of releasing a package)
*Property of live update possible *Property of live update possible
== Package Format == === Package Format ===
A package format has to meet the following requirements: A package format has to meet the following requirements:
- It must be able to store BeOS/Haiku file attributes. - It must be able to store BeOS/Haiku file attributes.
@@ -257,7 +256,7 @@ A package format has to meet the following requirements:
[PackageFormat The Haiku Package format] specifies a format meeting these requirements. [PackageFormat The Haiku Package format] specifies a format meeting these requirements.
== References == === References ===
(1) http://www.haiku-os.org/glass_elevator/rfc/installer (1) http://www.haiku-os.org/glass_elevator/rfc/installer
@@ -265,10 +264,10 @@ A package format has to meet the following requirements:
(3) http://www.freelists.org/post/haiku-development/software-organizationinstallation,8 and http://www.freelists.org/post/haiku-development/software-organizationinstallation,55 (3) http://www.freelists.org/post/haiku-development/software-organizationinstallation,8 and http://www.freelists.org/post/haiku-development/software-organizationinstallation,55
{{{ <pre>
#!div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;" #div style="background-color: #EEEEEE; border: 1px solid; margin: 20pt; padding: 5pt;"
'''jonas.kirilla''': I hope it's clear enough in reference 3 (above) that my ideas on package management approach it from a different angle. Which may or may not overlap with the use of libalpm. FWIW, I'm not ready to endorse all aspects of this proposal. '''jonas.kirilla''': I hope it's clear enough in reference 3 (above) that my ideas on package management approach it from a different angle. Which may or may not overlap with the use of libalpm. FWIW, I'm not ready to endorse all aspects of this proposal.
}}} </pre>
(4) http://www.freelists.org/post/haiku-development/Pathrelocatable-software-and-assigns (4) http://www.freelists.org/post/haiku-development/Pathrelocatable-software-and-assigns
+14 -13
View File
@@ -1,9 +1,10 @@
= Packaging Policy =
== Packaging Policy ==
This document defines the policy for creating packages. This document defines the policy for creating packages.
== Multiple Packages per Software == === Multiple Packages per Software ===
Installation files of a software shall be put into separated packages according to their purpose. For a software "foo": Installation files of a software shall be put into separated packages according to their purpose. For a software "foo":
@@ -18,7 +19,7 @@ If multiple packages are defined, they must not contain any common files.
If a software contains client and server software that can be used independently from each other, two separate sets of packages shall be created. If a software contains client and server software that can be used independently from each other, two separate sets of packages shall be created.
== Provides Declaration == === Provides Declaration ===
A package "foo" must declare the following provided resolvables: A package "foo" must declare the following provided resolvables:
@@ -30,7 +31,7 @@ A package "foo" must declare the following provided resolvables:
Any instance of '-' in <name> shall be replaced by '_'. If the backward compatibility of a resolvable is known, a "compat >= <compatibleVersion>" shall be added accordingly. Any instance of '-' in <name> shall be replaced by '_'. If the backward compatibility of a resolvable is known, a "compat >= <compatibleVersion>" shall be added accordingly.
== Documentation == === Documentation ===
If a package "foo" provides documentation (which it should, of course), in many cases that can be provided in different formats: If a package "foo" provides documentation (which it should, of course), in many cases that can be provided in different formats:
@@ -40,30 +41,30 @@ If a package "foo" provides documentation (which it should, of course), in many
- Other documentation for a package foo -- HTML, a simple ReadMe, sample documents, PDFs, etc. -- goes into subdirectory "packages/foo". If it is likely that multiple versions of a package may be installed, then a version string (as appropriate just major, major and minor, or even full (but no revision)) should be appended, e.g. "package/foo-2" or "package/foo-2.13". - Other documentation for a package foo -- HTML, a simple ReadMe, sample documents, PDFs, etc. -- goes into subdirectory "packages/foo". If it is likely that multiple versions of a package may be installed, then a version string (as appropriate just major, major and minor, or even full (but no revision)) should be appended, e.g. "package/foo-2" or "package/foo-2.13".
*For a package foo_devel developer documentation, except man and info pages, should go into "develop/documentation/foo". A version string may be appended to the directory name as well. When it is unclear what is developer documentation or it isn't really possible to separate it from user documentation "documentation/foo" should be used. *For a package foo_devel developer documentation, except man and info pages, should go into "develop/documentation/foo". A version string may be appended to the directory name as well. When it is unclear what is developer documentation or it isn't really possible to separate it from user documentation "documentation/foo" should be used.
== Data Files == === Data Files ===
Data files for a package foo shall generally be placed in a directory "data/foo". If it is likely that multiple versions of a package may be installed, then a version string shall be appended. Data (but not settings/configuration) files generated at run-time shall be placed in "cache/foo" or "var/foo", depending on the kind of data the files contain. For data files, both read-only or generated, that are shared between different packages/software a differently named subdirectory may be used as appropriate (e.g. font files are placed in "data/fonts"). Data files for a package foo shall generally be placed in a directory "data/foo". If it is likely that multiple versions of a package may be installed, then a version string shall be appended. Data (but not settings/configuration) files generated at run-time shall be placed in "cache/foo" or "var/foo", depending on the kind of data the files contain. For data files, both read-only or generated, that are shared between different packages/software a differently named subdirectory may be used as appropriate (e.g. font files are placed in "data/fonts").
== Writable and Settings Files and Directories == === Writable and Settings Files and Directories ===
All global writable files and directories as well as user settings files and directories that the package includes or the packaged software creates or requires the user to create shall be declared by the package (via GLOBAL_WRITABLE_FILES respectively USER_SETTINGS_FILES in the build recipe) in the following way: All global writable files and directories as well as user settings files and directories that the package includes or the packaged software creates or requires the user to create shall be declared by the package (via GLOBAL_WRITABLE_FILES respectively USER_SETTINGS_FILES in the build recipe) in the following way:
*A user specific settings file shall never be installed on package activation. Usually user specific settings files are completely optional. In the rare case that a software requires a user specific settings, the user will have to create it manually. In either case, if the package includes a template user settings file, that should be declared: *A user specific settings file shall never be installed on package activation. Usually user specific settings files are completely optional. In the rare case that a software requires a user specific settings, the user will have to create it manually. In either case, if the package includes a template user settings file, that should be declared:
{{{ <pre>
USER_SETTINGS_FILES="settings/foo template data/foo/user-settings-template" USER_SETTINGS_FILES="settings/foo template data/foo/user-settings-template"
}}} </pre>
If no template file is included, the settings file shall still be declared: If no template file is included, the settings file shall still be declared:
{{{ <pre>
USER_SETTINGS_FILES="settings/foo" USER_SETTINGS_FILES="settings/foo"
}}} </pre>
*Since many ported software requires a global settings file or other writable files, a default version of such a file can be provided and is automatically installed on package activation. In that case the package must also declare what shall be done with a user-modified file when the package is updated. E.g.: *Since many ported software requires a global settings file or other writable files, a default version of such a file can be provided and is automatically installed on package activation. In that case the package must also declare what shall be done with a user-modified file when the package is updated. E.g.:
{{{ <pre>
GLOBAL_WRITABLE_FILES="settings/foo keep-old" GLOBAL_WRITABLE_FILES="settings/foo keep-old"
}}} </pre>
"keep-old" indicates that the software can read old files and the user-modified file should be kept. "manual" indicates that the software may not be able to read an older file and the user may have to manually adjust it. "auto-merge" indicates that the file format is simple text and a three-way merge shall be attempted. If a default settings file is not included in the package, the settings file shall still be declared, just without the additional keyword. "keep-old" indicates that the software can read old files and the user-modified file should be kept. "manual" indicates that the software may not be able to read an older file and the user may have to manually adjust it. "auto-merge" indicates that the file format is simple text and a three-way merge shall be attempted. If a default settings file is not included in the package, the settings file shall still be declared, just without the additional keyword.
In both cases, user settings files and global writable files, the "directory" keyword can be used to indicate that the given path actually refers to a directory. In both cases, user settings files and global writable files, the "directory" keyword can be used to indicate that the given path actually refers to a directory.
== Post-Installation Scripts == === Post-Installation Scripts ===
A package may include one or more post-installation scripts. The scripts are executed whenever the package is activated (for the first time, but also after package updates). They shall be placed in "boot/post-install" and declared explicitly by the package (via POST_INSTALL_SCRIPTS in the build recipe). A post-install script should be considered the last resort. It should only be used, if there's no reasonable alternative. A package may include one or more post-installation scripts. The scripts are executed whenever the package is activated (for the first time, but also after package updates). They shall be placed in "boot/post-install" and declared explicitly by the package (via POST_INSTALL_SCRIPTS in the build recipe). A post-install script should be considered the last resort. It should only be used, if there's no reasonable alternative.
+26 -25
View File
@@ -1,34 +1,35 @@
These are the TODO items for the Haiku '''Package Management'''. These are the TODO items for the Haiku '''Package Management'''.
=== packagefs === ==== packagefs ====
* If necessary, add a caching mechanism to speed up mounting it. **If necessary, add a caching mechanism to speed up mounting it.
=== Package Daemon === ==== Package Daemon ====
* Complete support for extracting and updating settings files: Merge support and user feedback are still missing. **Complete support for extracting and updating settings files: Merge support and user feedback are still missing.
* Add user notification/interaction support for initial verification (on start-up). **Add user notification/interaction support for initial verification (on start-up).
* Support packages being copied to the `packages` directory. Currently only moving works. **Support packages being copied to the <tt>packages</tt> directory. Currently only moving works.
=== Package building === ==== Package building ====
* Define packaging guidelines and create a tool to check packages against those. **Define packaging guidelines and create a tool to check packages against those.
* Status: Here's the [wiki:PackageManagement/PackagingPolicy wiki page] defining the policy. It's still a work in progress. haikuporter has some policy checking built in already. ***Status: Here's the [[PackageManagement/PackagingPolicy| wiki page]] defining the policy. It's still a work in progress. haikuporter has some policy checking built in already.
=== Package kit/manager === ==== Package kit/manager ====
* Add system update support. **Add system update support.
* Status: Mostly functional in pkgman (`pkgman full-sync`), but unsupported in HaikuDepot. ***Status: Mostly functional in pkgman (<tt>pkgman full-sync</tt>), but unsupported in HaikuDepot.
=== Boot loader === ==== Boot loader ====
* Safe mode/recovery options: **Safe mode/recovery options:
- Disable packages installed in home. - Disable packages installed in home.
=== Package/package repository format === ==== Package/package repository format ====
* Add localization support. [http://www.freelists.org/post/haiku-depot-web/Title-localization,18 More] [http://www.freelists.org/post/haiku-depot-web/Title-localization,29 info]. **Add localization support. [http://www.freelists.org/post/haiku-depot-web/Title-localization,18 More] [http://www.freelists.org/post/haiku-depot-web/Title-localization,29 info].
* Add support for repository keys (public/private) and package signing/check sums, so that it is possible to verify that data retrieved from a repository have not been tampered with. **Add support for repository keys (public/private) and package signing/check sums, so that it is possible to verify that data retrieved from a repository have not been tampered with.
* Add package sizes to repository. Since there are plans to support xz (or other high-ratio formats) compressed uncompressed packages for download that are recompressed for installation, we probably need to discriminate between download and installation sizes. **Add package sizes to repository. Since there are plans to support xz (or other high-ratio formats) compressed uncompressed packages for download that are recompressed for installation, we probably need to discriminate between download and installation sizes.
* Add support for a faster compression format (e.g. [https://code.google.com/p/lz4/ lz4]). As it turns out, the currently used zlib compression is rather slow (slower than reading uncompressed data from a slow HD). **Add support for a faster compression format (e.g. [https://code.google.com/p/lz4/ lz4]). As it turns out, the currently used zlib compression is rather slow (slower than reading uncompressed data from a slow HD).
* Add MIME info for types supported by packaged applications (and sniffer rules?) to the package attributes. **Add MIME info for types supported by packaged applications (and sniffer rules?) to the package attributes.
=== Miscellaneous === ==== Miscellaneous ====
* Add MIME directory watching to registrar, so it can update the MIME DB caches when packages are de-/activated. **Add MIME directory watching to registrar, so it can update the MIME DB caches when packages are de-/activated.
* Get rid of post_install scripts. Do that in the build system. **Get rid of post_install scripts. Do that in the build system.
* Get rid of UserBootscript. Start the scripts in "~/config/settings/boot/launch" from Bootscript instead. **Get rid of UserBootscript. Start the scripts in "~/config/settings/boot/launch" from Bootscript instead.
* Update Installer. **Update Installer.
+14 -13
View File
@@ -1,20 +1,21 @@
= Package Management =
== Package Management ==
This is a short index of the available package management related documentation. This is a short index of the available package management related documentation.
* [wiki:PackageManagement/Infrastructure Package Management Infrastructure] provides an overview of what components belong to Haiku's package management infrastructure and how they work and interact. *[[PackageManagement/Infrastructure| Package Management Infrastructure]] provides an overview of what components belong to Haiku's package management infrastructure and how they work and interact.
* [wiki:PackageManagement/BuildingPackages Building Packages] gives information on various aspects of the package building process. *[[PackageManagement/BuildingPackages| Building Packages]] gives information on various aspects of the package building process.
* [wiki:PackageManagement/DirectoryStructure Boot Volume Directory Structure] outlines the directory structure of a package management powered Haiku boot volume. *[[PackageManagement/DirectoryStructure| Boot Volume Directory Structure]] outlines the directory structure of a package management powered Haiku boot volume.
* [wiki:PackageManagement/FileFormat Haiku Package File Format] specifies in detail the file format of Haiku package files (HPKG) and Haiku package repository files (HPKR). *[[PackageManagement/FileFormat| Haiku Package File Format]] specifies in detail the file format of Haiku package files (HPKG) and Haiku package repository files (HPKR).
* [wiki:PackageManagement/PackagingPolicy Packaging Policy] defines the policy for creating Haiku packages. *[[PackageManagement/PackagingPolicy| Packaging Policy]] defines the policy for creating Haiku packages.
* [wiki:PackageManagement/PackagesStatus Packages Status] gives an overview of the packages needed for a Haiku standard build and their current status. *[[PackageManagement/PackagesStatus| Packages Status]] gives an overview of the packages needed for a Haiku standard build and their current status.
* [wiki:PackageManagement/HybridBuilds Hybrid Builds] provides some information regarding hybrid builds. *[[PackageManagement/HybridBuilds| Hybrid Builds]] provides some information regarding hybrid builds.
* [wiki:PackageManagement/HaikuDepot HaikuDepot] explains the concept and ideas behind HaikuDepot, the GUI package manager in the making. *[[PackageManagement/HaikuDepot| HaikuDepot]] explains the concept and ideas behind HaikuDepot, the GUI package manager in the making.
* [wiki:PackageManagement/Migration Migration] lists the changes that users should expect when migrating to a package management Haiku. *[[PackageManagement/Migration| Migration]] lists the changes that users should expect when migrating to a package management Haiku.
* [wiki:PackageManagement/Bootstrapping Bootstrapping Haiku] explains the process of bootstrapping Haiku and third-party packages. *[[PackageManagement/Bootstrapping| Bootstrapping Haiku]] explains the process of bootstrapping Haiku and third-party packages.
* [wiki:PackageManagement/TODO TODO] is a list of package management related work still to be done. *[[PackageManagement/TODO| TODO]] is a list of package management related work still to be done.
*[https://www.haiku-os.org/tags/package-management/ Blog posts] on package management (the Batisseur ones are only indirectly package management related). *[https://www.haiku-os.org/tags/package-management/ Blog posts] on package management (the Batisseur ones are only indirectly package management related).
* [wiki:PackageManagement/OldIdeas (Old) Package Management Ideas] is a collection of thoughts and discussions regarding package management. It has been partially obsoleted by the progress on the package management implementation. *[[PackageManagement/OldIdeas| (Old) Package Management Ideas]] is a collection of thoughts and discussions regarding package management. It has been partially obsoleted by the progress on the package management implementation.
*http://www.youtube.com/watch?v=rNZQQM5zU-Q&list=PL3FFCD4C6D384A302 is a video playlist of Ingo and Oliver explaining and demonstrating the package management branch at BeGeistert 2011. *http://www.youtube.com/watch?v=rNZQQM5zU-Q&list=PL3FFCD4C6D384A302 is a video playlist of Ingo and Oliver explaining and demonstrating the package management branch at BeGeistert 2011.