# pybuild package

## Module contents

The module accessible within pybuilds

Note that this module should not be imported outside of pybuild files

class pybuild.File(NAME, REQUIRED_USE='', OVERRIDES=[], **kwargs)[source]

Bases: object

Represents important installed files and their metadata

Warning

This class has been deprecated as of Portmod 2.4. It will be removed in Portmod 3.0

Parameters
NAME: str

Name of the file relative to the root of the InstallDir

OVERRIDES: Union[str, List[str]]

A list of files which this overrides when sorting (if applicable).

Can either be in the form of a string containing use-conditionals (note that this does not support files that contain spaces) or a list of files to override. Note that these overridden files are not considered masters and do not need to be present.

For archives it determines the order in which the fallback archives will be searched during VFS lookups.

REQUIRED_USE: str

Requirements for installing this file

The default empty string is always satisfied. See Pybuild2.REQUIRED_USE for details on the syntax.

class pybuild.InstallDir(PATH, REQUIRED_USE='', PATCHDIR='.', S=None, WHITELIST=None, BLACKLIST=None, RENAME=None, DATA_OVERRIDES='', ARCHIVES=(), VFS=None, DOC=(), **kwargs)[source]

Bases: object

Represents a directory in the Virtual File System

Note that arbitrary arguments can be passed to the constructor, as repositories may make use of custom information. See the repository-level documentation for such information.

Warning

This class has been deprecated as of Portmod 2.4. It will be removed in Portmod 3.0

Parameters
ARCHIVES: List[File]

A list of File objects representing VFS archives.

These will be searched, in order, during VFS file lookups if the file is not present in the package directories.

BLACKLIST: Optional[List[str]]

If present, does not install files matching the patterns in this list. fnmatch-style globbing patterns (e.g. * and [a-z]) can be used

DATA_OVERRIDES: str

A list of packages that this InstallDir should override in the VFS

This only has a different effect from Pybuild1.DATA_OVERRIDES if multiple PATCHDIRs are set, as it can define overrides for individual PATCHDIRS, while Pybuild1.DATA_OVERRIDES affects all PATCHDIRs. See Pybuild1.DATA_OVERRIDES for details of the syntax.

DOC: List[str]

A list of patterns matching documentation files within the package

This documentation will be installed separately fnmatch-style globbing patterns (e.g. * and [a-z]) can be used.

PATCHDIR: str

The destination path of the InstallDir within the package’s directory.

Defaults to “.”, i.e. the root of the mod directory. If multiple InstallDirs share the same PATCHDIR they will be installed into the same directory in the order that they are defined in the INSTALL_DIRS list. Each unique PATCHDIR has its own entry in the VFS, and its own sorting rules

PATH: str

The path to the data directory that this InstallDir represents relative to the root of the archive it is contained within.

RENAME: Optional[str]

Destination path of this directory within the final directory.

E.g.:

InstallDir("foo/bar", PATCHDIR=".", RENAME="bar")


Will install the contents of foo/bar (in the source) into the directory bar inside the package’s installation directory (and also the VFS).

REQUIRED_USE: str

A list of use flags with the same format as the package’s REQUIRED_USE variable which enable the InstallDir if satisfied. Defaults to an empty string that is always satisfied.

S: Optional[str]

The source directory corresponding to this InstallDir.

Similar function to S for the entire pybuild, this determines which directory contains this InstallDir, and generally corresponds to the name of the source archive, minus extensions. This is required for packages that contain more than one source, but is automatically detected for those with only one source if it is not specified, and will first take the value of Pybuild2.S, then the source’s file name without extension if the former was not defined.

VFS: Optional[bool]

Whether or not this InstallDir gets added to the VFS

Defaults to the value of the VFS variable in the profile configuration

WHITELIST: Optional[List[str]]

If present, only installs files matching the patterns in this list. fnmatch-style globbing patterns (e.g. * and [a-z]) can be used

get_files()[source]

Generator function yielding file subattributes of the installdir

class pybuild.Pybuild1[source]

Bases: Pybuild2

Legacy class. Superseded by Pybuild2

Warning

This class has been deprecated as of Portmod 2.4. It will be removed in Portmod 3.0

DATA_OVERRIDES = ''

A use-reduce-able list of atoms indicating packages whose data directories should come before the data directories of this package when sorting data directories.

They do not need to be dependencies. Blockers (atoms beginning with !!) can be used to specify underrides, and use dependencies (e.g. the [bar] in foo[bar]) can be used to conditionally override based on the target atom’s flag configuration.

Not included in PMS

INSTALL_DIRS: List[InstallDir] = []

The INSTALL_DIRS variable consists of a python list of InstallDir objects.

E.g.:

INSTALL_DIRS=[
InstallDir(
'Morrowind/Data Files',
REQUIRED_USE='use use ...',
DESTPATH='.',
PLUGINS=[File('Plugin Name',
REQUIRED_USE='use use ...', satisfied
)],
ARCHIVES=[File('Archive Name')],
S='Source Name Without Extension',
)
]


Not included in PMS

REBUILD_FILES: List[str] = []

Files in the VFS which, if changed, should cause this package to be rebuilt

Can include glob-style patterns using the , ? and [] operators. See https://docs.python.org/3/library/fnmatch.html. Unlike normal fnmatch parsing, wildcards () will not match accross path separators.

This field can be modified during installation, and will only be used after the package has been installed.

TIER = 'a'

The Tier of a package represents the position of its data directories and plugins in the virtual file system.

This is used to group packages in such a way to avoid having to individually specify overrides whenever possible.

The value is either in the range [0-9] or [a-z].

Default value: ‘a’

Tier 0 represents top-level mods such as morrowind Tier 1 is for mods that replace or modify top-level mods. E.g. texture and mesh replacers. Tier 2 is for large mods that are designed to be built on top of by other mods, such as Tamriel Data Tier a is for all other mods. Tier z is for mods that should be installed or loaded last. E.g. omwllf The remaining tiers are reserved in case the tier system needs to be expanded

Not included in PMS

get_files(typ)[source]

Returns all enabled files and their directories

Parameters

typ (str) –

Return type
src_install()[source]

The src_install function installs the package’s content to a directory specified in Pybuild2.D.

The initial working directory is Pybuild2.S, falling back to Pybuild.WORKDIR if the directory does not exist.

The default implementation used when the package lacks the src_install function moves each InstallDir in Pybuild1.INSTALL_DIRS which is not hidden due to an unsatisfied REQUIRED_USE into Pybuild2.D.

unpack(archives)[source]

Unpacks the given archive into the workdir

Uses patool as its backend.

Parameters

archives (Union[str, Iterable[Union[Source, str]]]) – The list of archives to be unpacked

class pybuild.Pybuild2[source]

Bases: FullPybuild

The class all Pybuilds should derive.

The name and path of a pybuild declares the package name, category, version and (optionally) revision:

{CATEGORY}/{PKG_NAME}-{VER}(-r{REV}).pybuild


Categories and package names may contain lower case letters, numbers and hyphens. Versions may contain numbers and dots. Revisions may only contain numbers (following the -r prefix). (See the PMS for the complete naming scheme).

Note that revisions refer to revisions of the pybuild itself, not the package, and are used to indicate that the way the mod is configured has changed in a way that will impact the installed version. For changes, such as the source files moving, that would not impact a mod that is already installed, you do not need to update the revision.

There are certain fields which are defined automatically and may only be available in some scopes:

Variable

Scope

Pybuild2.P

All scopes

Pybuild2.PF

All scopes

Pybuild2.PN

All scopes

Pybuild2.CATEGORY

All scopes

Pybuild2.PV

All scopes

Pybuild2.PVR

All scopes

Pybuild2.USE

All scopes except __init__

Pybuild2.WORKDIR

src_*

Pybuild2.T

All scopes except __init__

Pybuild2.D

Pybuild2.src_install()

Pybuild2.FILESDIR

src_*

Pybuild2.ROOT

src_*, pkg_*

Pybuild2.A

src_*, pkg_nofetch

Pybuild2.UNFETCHED

pkg_nofetch

Pybuild2.S

src_*

1(1,2,3,4,5,6,7,8,9,10,11,12,13)

Fields which are set automatically and should not be defined in the package file. Described in the table above.

A: List[Source]

The list of enabled sources 1

Scope: All except __init__

CATEGORY: str

The package’s category. 1

E.g. base

D: str

The full path of the directory where the package is to be installed. 1

Note that this is a temporary directory and not the final install location.

Scope: src_install

DEPEND: str = ''

Build dependencies.

The DEPEND field is used to specify packages which need to be installed in order for this package to install correctly.

Most mods do not have build dependencies, however mods that require patching using tools external to portmod, or packages that generate content from other sources, will need to include their masters, or the other sources, as build dependencies, to ensure that they are installed prior to the package being installed.

Format (both DEPEND and RDEPEND): A list of dependencies in the form of package atoms. All dependencies should include both category and package name. Versions should also be included if the package depends on a specific version of another mod. It is recommended not to include a version number in the dependency unless it is known that the package will not work with other versions.

Ranges of versions can be indicated by prefixing >,<,<=,>= to the atoms. E.g. >=cat/foo-1.0

Specific versions can be indicated by prefixing = (matches version and revision exactly) or ~ (matches version, but allows any revision) to the atoms. E.g. =cat/foo-1.0

Use flag dependencies can be specified in the following manner:

• cat/foo[flag] - Indicates that flag must be enabled

• cat/foo[flag,flag2] - Indicates that both flag and flag2 must be enabled

• cat/foo[!flag] - Indicates that flag must be disabled

• cat/foo[flag?] - Indicates that flag must be enabled if it is enabled for this package

• cat/foo[!flag?] - Indicates that flag must be disabled if it is enabled for this package

Atoms can be surrounded by use-conditionals if they are only dependencies when that use flag is enabled/disabled.

E.g. flag? ( cat/foo )

Atoms can be grouped and prefixed by a || operator to indicate that any of the given packages will satisfy the dependency.

E.g. || ( cat/foo cat/bar cat/baz )

Note that it is required that the parentheses ( ) are separated from the atoms by whitespace.

Packages which cannot be installed at the same time can be marked as blocks using the !! operator. I.e. !!cat/foo indicates that cat/foo cannot be installed at the same time as the current package.

DESC: str = ''

A short description of the package.

Is may (depending on options provided) be used in searches. Note that a longer description can be provided in metadata.yaml.

DOCS: List[str] = ['README*', 'readme*', 'ReadMe*', 'ChangeLog', 'CHANGELOG*', 'AUTHORS*', 'NEWS*', 'TODO*', 'CHANGES*', 'THANKS*', 'BUGS*', 'FAQ*', 'CREDITS*', 'Doc/*', 'doc/*', 'docs/*', 'Docs/*']

A list of documentation patterns for the default src_install to install using Pybuild2.dodoc

FILESDIR: str

Path of the directory containing additional repository files

Scope: src_*

HOMEPAGE: str = ''

The URL of the package’s homepage(s).

Used for descriptive purposes and included in search results.

IUSE: Set[str] = ''

A field containing a space-separated list of the use flags used by the package.

IUSE should contain all regular use flags used by this package, both local and global. Prefixing the use flags with a + means that the option is enabled by default. Otherwise use flags are disabled by default.

Note that you do not need to include TEXTURE_SIZES type flags in IUSE, but USE_EXPAND variables should be included in IUSE.

KEYWORDS: str = ''

Keywords indicating compatibility. Existence of the keyword indicates that the mod is stable on that platform.

a ~ in front of the keyword indicates that the mod is unstable on that platform no keyword indicates that the mod is untested on that platform a - in front of the keyword indicates that the mod is known to not work on that platform

E.g. A package that works on OpenMW but does not on tes3mp:

KEYWORDS='openmw -tes3mp'

LICENSE: str = ''

One or more licenses used by the package.

A list of licenses can be found in the licenses directory of the repository.

NAME: str = ''

Descriptive package name.

The package name used for identification is the name used in the filename, however this name is included when searching for packages.

P: Atom

The package name and version. 1

E.g.: example-suite-1.0

PATCHES: str = ''

A list of patch files stored within the package’s files directory in the repository

Note that unlike as specified in the PMS, their paths must be relative to the files directory.

See apply_patch() for details on the supported patch format.

PF: Atom

The package name with version and revision. 1

E.g.: example-suite-1.0-r1

PN: Atom

The package name without version. 1

E.g.: example-suite

PR: str

The package’s revision 1

E.g. r1

Is equal to r0 is no revision is specified

PROPERTIES: str = ''

A white-space-delimited list of additional properties of the given pybuild to enable special behaviour.

Possible values are given below:

• live: Indicates that the pybuild doesn’t have a specific version (e.g. if

installing from a git repository branch but not using a specific commit). Live pybuilds should have an empty KEYWORDS list, as stability testing is not meaningful if the upstream source is changing. Live packages must override Pybuild2.can_update_live().

• local: Only used internally to refer to Local mods with generated metadata

PV: str

The package’s version without revision 1

E.g. 1.0

PVR: str

The package’s version and revision 1

E.g. 1.0-r1

RDEPEND: str = ''

Runtime dependencies.

Is used to specify packages which are required at runtime for this package to function. The format is the same as for DEPEND

REQUIRED_USE: str = ''

An expression indicating valid combinations of use flags.

Consists of a string containing sub-expressions of the form given below. Note that the brackets can contain arbitrary nested expressions of this form, and are not limited to what is shown in the examples below.

Behaviour

Expression

flag must be enabled

flag

flag must not be enabled

!flag

If flag1 enabled then flag2 enabled

flag1? ( flag2 )

If flag1 disabled then flag2 enabled

!flag1? ( flag2 )

If flag1 disabled then flag2 disabled

!flag1? ( !flag2 )

Must enable any one or more (inclusive or)

|| ( flag1 flag2 flag3 )

Must enable exactly one but not more (exclusive or)

^^ ( flag1 flag2 flag3 )

May enable at most one

?? ( flag1 flag2 flag3 )

RESTRICT: str = ''

Lists features which should be disabled for this package

The following two options are supported:

• mirror: The package’s SRC_URI entries should not be mirrored, and mirrors should not be checked when fetching.

• fetch: The packages’s SRC_URI entries should not be fetched automatically, and the pkg_nofetch function should be invoked if a source cannot be found. This option implies mirror.

Note that portmod also supports determining these automatically based on source URIs and licenses, so it is no longer necessary to set them explicitly. mirror is restricted for licenses which are not in the REDISTRIBUTABLE license group (see license_groups.yaml), and fetch is restricted for files which are not redistributable (according to license) and do not have a scheme in their SRC_URI (i.e. just a filename, no https://domain.tld etc.).

ROOT: str

The full path of the prefix root where packages will be installed

Note: This functions as both ROOT and SYSROOT (as defined by PMS section 11.1).

Scope: src_*, pkg_*

S: Optional[str] = None

Specifies the default working directory for src_* functions.

The default value (if S is None) is the name (minus extension) of the first source in SRC_URI (after use-conditionals have been evaluated). If this path does not exist, the working directory falls back to WORKDIR.

This is also used to determine the base source path used for installing a InstallDir in the default src_install if S is not defined on the InstallDir.

SRC_URI: str = ''

A List of sources to be fetched.

If source files should be renamed, this can be done with the arrow operator as shown in the example below.

Sources can be wrapped in use-conditional expressions to prevent certain sources from being downloaded unless certain use flags are set or unset.

E.g.:

SRC_URI="""
http://mw.modhistory.com/file.php?id=9321 -> FileName-1.0.zip
flag? ( https://cdn.bethsoft.com/elderscrolls/morrowind/other/masterindex.zip )
"""


Note that if you are renaming files, they should correspond to the original filename as best possible, but should also contain version information of some sort to prevent conflicts with other sources from the same package. That is, if the package is updated, we do not want the updated source name to be the same as a previous source name, even if the source name did not change upstream.

T: str

Path to a temporary directory which may be used during packaging 1

Scope: All except __init__

TEXTURE_SIZES: str = ''

A field declaring the texture size options that the package supports.

If only one texture size option is available, this field need not be included. Texture sizes should be numbers representing the size of the texture in pixels. Given that textures are usually two-dimensional, the convention is to use: $$\sqrt{ l \cdot w}$$

E.g.:

TEXTURE_SIZES = "1024 2048"


This is a special type of USE_EXPAND variable, as use flags are created for its values in the form texture_size_SIZE (in the above example texture_size_1024 and texture_size_2048).

These use flags can (and should) be used in the pybuild to enable sources and InstallDirs conditionally depending on whether or not the texture size was selected. Exactly one of these use flags will be enabled when the mod is installed depending on the value of the TEXTURE_SIZE variable in the user’s portmod.cfg.

Not included in the PMS

UNFETCHED: List[Source]

The list of sources which need to be fetched 1

Scope: pkg_nofetch

USE: Set[str]

Enabled use flags 1

Scope: All except __init__

WORKDIR: str

The directory where packaging takes place 1

Scope: src_*

can_update_live()[source]

Indicates whether or not a live package can be updated.

The default implementation just returns False. If the package has live in its PROPERTIES, it must implement this method.

Return type

bool

Returns

If the package has PROPERTIES="live" and can be updated, should return True Otherwise, should return False

dodoc(pattern)[source]

Installs documentation matching the given pattern into the image directory (Self.D)

Parameters

pattern (str) – A pattern which can include glob-style wildcards as implemented by glob.

static execute(command, pipe_output=False, pipe_error=False)[source]

Allows execution of arbitrary commands at runtime. Command is sandboxed with filesystem and network access depending on the context in which it is called

Parameters
Return type
get_installed_env()[source]

Returns a dictionary containing installed object values

info(string)[source]

Displays info message both immediately, and in the summary after all transactions have been completed

Parameters

string (str) – String to display

pkg_postinst()[source]

Function called immediately after package installation

In Pybuild1, this function has full write permissions to ROOT. In Pybuild2 it only has read permissions.

Note that the default does nothing, and it will not even be executed unless defined.

pkg_prerm()[source]

Function called immediately before package removal

In Pybuild1, this function has full write permissions to ROOT. In Pybuild2 it only has read permissions.

Note that the default does nothing, and it will not even be executed unless defined.

pkg_pretend()[source]

May be used to carry out sanity checks early on in the install process

Note that the default does nothing, and it will not even be executed unless defined.

pkg_pretend is run separately from the main phase function sequence, and does not participate in any kind of environment saving. There is no guarantee that any of an package’s dependencies will be met at this stage, and no guarantee that the system state will not have changed substantially before the next phase is executed.

pkg_pretend must not write to the filesystem and the initial working directory should not be expected to be consistent.

src_install()[source]

The src_install function installs the package’s content to a directory specified in Pybuild2.D.

The initial working directory is Pybuild2.S, falling back to Pybuild.WORKDIR if the directory does not exist.

The default implementation used when the package lacks the src_install function shall behave as:

for pattern in self.DOCS:
self.dodoc(pattern)

src_prepare()[source]

The src_prepare function can be used for post-unpack source preparation.

The initial working directory is Pybuild2.S, falling back to Pybuild2.WORKDIR if the directory does not exist.

The default implementation used when the package lacks the src_prepare function shall behave as:

if self.PATCHES:
for patch in use_reduce(self.PATCHES, self.USE, flat=True):
path = os.path.join(self.FILESDIR, patch)
apply_patch(path)

src_unpack()[source]

The src_unpack function extracts all of the package’s sources.

The initial working directory must be self.WORKDIR, and the default implementation used when the package lacks the src_unpack function shall behave as:

self.unpack(self.A)

unpack(archives)[source]

Unpacks the given archive into the workdir

Uses shutil.unpack_archive() as its backend, supporting the following archive formats: - .zip - .tar - .tar.xz / .txz - .tar.bz2 / .tbz2 - .tar.xz / .txz

Parameters

archives (Union[str, Iterable[Union[Source, str]]]) – The list of archives to be unpacked

validate()[source]

(Since Portmod 2.4) inquisitor will call this function when scanning repositories.

Only code that would be valid in the package global scope (see Sandbox) may be used.

This is designed to allow basic structural checks without hindering package loading.

It differs from Pybuild2.pkg_pretend() in that it is meant for static checks, not runtime checks.

warn(string)[source]

Displays warning message both immediately, and in the summary after all transactions have been completed

Parameters

string (str) – String to display

pybuild.apply_patch(patch)[source]

Applies git patch using Git apply

Patch files must be in a format that can be applied via git apply. Such patches can be produced with git diff --no-index ORIG NEW. The --binary option can be used to produce binary diffs for non-text files.

Patches must be self-applying. I.e. they should not rely on paths being passed to git apply, and must apply from the default working directory in src_prepare.

It is recommended that a comment header is included to describe what the patch does, where it’s from etc.

Parameters

patch (str) – Path to the patch to be applied

pybuild.find_file(name)[source]

Locates the path of a file within the OpenMW virtual file system

Warning

This function has been deprecated as of Portmod 2.4. It will be removed in Portmod 3.0

Parameters

name (str) – The relative path within the VFS to search for

Return type

str

Returns

The absolute path of the file

pybuild.get_masters(file)[source]

Detects masters for the given file

Warning

This function has been deprecated as of Portmod 2.4 it will be removed in Portmod 3.0

Parameters

file (str) – File to be examined

Return type
Returns

A set of all the master names

pybuild.list_dir(name)[source]

Locates all path of files matching the given pattern within the OpenMW virtual file system

Warning

This function has been deprecated as of Portmod 2.4. It will be removed in Portmod 3.0

Parameters

name (str) – The relative path of the directory within the VFS

Return type
Returns

A list of files contained within the directory

pybuild.patch_dir(src, dst, *, overwrite=True, ignore=None, case_sensitive=True, move_function=<function _move2>)[source]

Copies src ontop of dst

Parameters
Raises
Return type

str

Returns

Returns dst

pybuild.use_reduce(depstr, uselist={}, masklist={}, matchall=False, excludeall={}, is_src_uri=False, opconvert=False, flat=False, is_valid_flag=None, token_class=None, matchnone=False)[source]

Takes a dep string and reduces the use? conditionals out, leaving an array with subarrays. All redundant brackets are removed. Adapted from portage’s use_reduce

Parameters
Return type

List

Returns

The use reduced depend array

pybuild.version_gt(version1, version2)[source]

Version comparision function

Parameters
Return type

bool`

Returns

True if and only if version1 is greater than version2