Package commands#
Package definition files (package.py
) usually define a commands()
section. This is a python
function that determines how the environment is configured in order to include the package.
Consider the simple example:
def commands():
env.PYTHONPATH.append("{root}/python")
env.PATH.append("{root}/bin")
This is a typical case, where a package adds its source path to PYTHONPATH
, and its tools to
PATH
. The {root}
string expands to the installation directory of the package.
When a rez environment is configured, every package in the resolve list has its commands()
section
interpreted and converted into shell code (the language, bash or other, depends on the platform
and is extensible). The resulting shell code is sourced, and this configures the environment.
Within a configured environment, the variable REZ_CONTEXT_FILE
points at this shell code, and the
command rez-context --interpret
prints it.
The python API that you use in the commands()
section is called rex
(Rez EXecution language). It
is an API for performing shell operations in a shell-agnostic way. Some common operations you would
perform with this API include setting environment variables, and appending/prepending path-like
environment variables.
Note
By default, environment variables that are not referenced by any package are left unaltered. There will typically be many system variables that are left unchanged.
Warning
If you need to import any python modules to use in a commands()
section, the import statements must be done inside that function.
Order Of Command Execution#
The order in which package commands are interpreted depends on two factors: the order in which the packages were requested, and dependencies between packages. This order can be defined as:
If package
A
was requested before packageB
, thenA
’s commands are interpreted beforeB
’s;Unless package
A
requires (depends on)B
, in which caseB
will be interpreted beforeA
.
Consider a package maya_anim_tool
. Let us say this is a maya plugin. Naturally it has a dependency
on maya
, therefore maya
’s commands will be interpreted first. This is because the maya plugin
may depend on certain environment variables that maya
sets. For example, maya
might initialize
the MAYA_PLUG_IN_PATH
environment variable, and maya_anim_tool
may then append to this
variable.
For example, consider the request:
]$ rez-env maya_anim_tool-1.3+ PyYAML-3.10 maya-2015
Assuming that PyYAML
depends on python
, and maya_anim_tool
depends on maya
, then the
resulting commands()
execution order would be:
maya;
maya_anim_tool;
python;
PyYAML.
Variable Appending And Prepending#
Path-like environment variables can be appended and prepended like so:
env.PATH.append("{root}/bin")
However, the first append/prepend operation on any given variable actually overwrites the
variable, rather than appending. Why does this happen? Consider PYTHONPATH
: if an initial
overwrite did not happen, then any modules visible on PYTHONPATH
before the rez environment was
configured would still be there. This would mean you may not have a properly configured
environment. If your system PyQt
were on PYTHONPATH
for example, and you used rez-env to set
a different PyQt
version, an attempt to import it within the configured environment would still,
incorrectly, import the system version.
Note
PATH
is a special case. It is not simply overwritten, because if that
happened you would lose important system paths and thus utilities like ls
and cd
. In this
case the system paths are appended back to PATH
after all commands are interpreted. The system
paths are defined as the default value of PATH
in a non-interactive shell.
Noteasd
Better control over environment variable initialization is coming. Specifically, you will be able to specify various modes for variables. For example, one mode will append the original (pre-rez) value back to the resulting value.
String Expansion#
Object Expansion#
Any of the objects available to you in a commands()
section can be referred to in formatted strings
that are passed to rex functions such as setenv()
and so on. For example, consider the code:
appendenv("PATH", "{root}/bin")
Here, {root}
will expand out to the value of root
, which is the installation path of the
package (this.root
could also have been used).
You don’t have to use this feature. It is provided as a convenience. For example, the following code is equivalent to the previous example, and is just as valid (but more verbose):
import os.path
appendenv("PATH", os.path.join(root, "bin"))
Object string expansion is also supported when setting an environment variable via the env
object:
env.FOO_LIC = "{this.root}/lic"
Environment Variable Expansion#
Environment variable expansion is also supported when passed to rex functions. Both syntax $FOO
and ${FOO}
are supported, regardless of the syntax supported by the target shell.
Literal Strings#
You can use the literal()
function to inhibit object and environment variable string
expansion. For example, the following code will set the environment variable to the literal string:
env.TEST = literal("this {root} will not expand")
There is also an expandable()
function, which matches the default behavior. You wouldn’t typically
use this function. However, you can define a string containing literal and expandable parts by
chaining together literal()
and expandable()
:
env.DESC = literal("the value of {root} is").expandable("{root}")
Explicit String Expansion#
Object string expansion usually occurs only when a string is passed to a rex function, or to
the env
object. For example the simple statement var = "{root}/bin"
would not expand {root}
into var
. However, you can use the expandvars()
function to enable this behavior
explicitly:
var = expandvars("{root}/bin")
The expandvars()
and expandable()
functions are slightly different. expandable()
will generate a
shell variable assignment that will expand out while expandvars()
will expand the value immediately.
This table illustrates the difference between literal()
, expandable()
and expandvars()
:
Package command |
Equivalent bash command |
---|---|
|
|
|
|
|
|
Additional context
In Bash, single quote strings ('foo'
) will not be expanded.
Filepaths#
Rez expects POSIX-style filepath syntax in package commands, regardless of the shell or platform. Thus, even if you’re on Windows, you should do this:
def commands():
env.PATH.append("{root}/bin") # note the forward slash
Where necessary, filepaths will be automatically normalized for you. That is, converted into
the syntax expected by the shell. In order for this to work correctly however, rez needs to know
what environment variables are actually paths. You determine this with the
pathed_env_vars
config setting. By default, any environment
variable ending in PATH
will be treated as a filepath or list of filepaths, and any
set/append/prepend operation on it will cause those values to be path-normalized automatically.
Warning
Avoid using os.pathsep
or hardcoded lists of paths such as
{root}/foo:{root}/bah
. Doing so can cause your package to be incompatible with some shells or
platforms. Even the seemingly innocuous os.pathsep
is an issue, because there are some cases
(eg Git for Windows, aka git-bash) where the shell’s path separator does not match the underlying
system’s.
Pre And Post Commands#
Occasionally, it’s useful for a package to run commands either before or after all other packages,
regardless of the command execution order rules. This can be achieved by defining a pre_commands()
or post_commands()
function. A package can have any, all or none of pre_commands()
, commands()
and
post_commands()
defined, although it is very common for a package to define just commands()
.
The order of command execution is:
All package
pre_commands()
are executed, in standard execution order;Then, all package
commands()
are executed, in standard execution order;Then, all package
post_commands()
are executed, in standard execution order.
Pre Build Commands#
If a package is being built, that package’s commands are not run, simply because that package is not present in its own build environment! However, sometimes there is a need to run commands specifically for the package being built. For example, you may wish to set some environment variables to pass information along to the build system.
The pre_build_commands()
function does just this. It is called prior to the build. Note that info
about the current build (such as the installation path) is available in a
build
object (other commands functions do not have this object visible).
Pre Test Commands#
Sometimes it’s useful to perform some extra configuration in the environment that a package’s test
will run in. You can define the pre_test_commands()
function to do this. It will be invoked just
before the test is run. As well as the standard this
object, a test
object is also
provided to distinguish which test is about to run.
A Largish Example#
Here is an example of a package definition with a fairly lengthy commands()
section:
name = "foo"
version = "1.0.0"
requires = [
"python-2.7",
"~maya-2015"
]
def commands():
import os.path # imports MUST be inline to the function
# add python module, executables
env.PYTHONPATH.append("{this.root}/python")
env.PATH.append("{this.root}/bin")
# show include path if a build is occurring
if building:
env.FOO_INCLUDE_PATH = "{this.root}/include"
# debug support to point at local config
if defined("DEBUG_FOO"):
conf_file = os.path.expanduser("~/.foo/config")
else:
conf_file = "{this.root}/config"
env.FOO_CONFIG_FILE = conf_file
# if maya is in use then include the maya plugin part of this package
if "maya" in resolve:
env.MAYA_PLUG_IN_PATH.append("{this.root}/maya/plugins")
if resolve.maya.version.minor == "sp3":
error("known issue with GL renderer in service pack 3, beware")
# license file per major version
env.FOO_LIC = "/lic/foo_{this.version.major}.lic"
Objects#
Various objects and functions are available to use in the commands()
function (as well as
pre_commands()
and post_commands()
).
Following is a list of the objects and functions available.
- alias()#
Create a command alias.
alias("nukex", "Nuke -x")
Note
In
bash
, aliases are implemented as bash functions.
- build#
This is a dict like object. Each key can also be accessed as attributes.
This object is only available in the
pre_build_commands()
function. It has the following fields:if build.install: info("An installation is taking place") if build['build_type'] == 'local': pass
- build.build_type: Literal['local', 'central']#
One of
local
,central
. The type iscentral
if a package release is occurring, andlocal
otherwise.
- build.build_path: str#
Path to the build directory (not the installation path). This will typically reside somewhere within the
./build
subdirectory of the package being built.
- build.install_path: str#
Installation directory. Note that this will be set, even if an installation is not taking place.
Warning
Do not check this variable to detect if an installation is occurring. Use
build.install
instead.
- building: bool#
This boolean variable is
True
if a build is occurring (typically done via the rez-build tool), andFalse
otherwise.However, the
commands()
block is only executed when the package is brought into a resolved environment, so this is not used when the package itself is building. Typically a package will use this variable to set environment variables that are only useful during when other packages are being built. C++ header include paths are a good example.if building: env.FOO_INCLUDE_PATH = "{root}/include"
- command(arg: str)#
Run an arbitrary shell command.
Example:
command("rm -rf ~/.foo_plugin")
Note
Note that you cannot return a value from this function call, because the command has not yet run. All of the packages in a resolve only have their commands executed after all packages have been interpreted and converted to the target shell language. Therefore any value returned from the command, or any side effect the command has, is not visible to any package.
You should prefer to perform simple operations (such as file manipulations and so on) in python where possible instead. Not only does that take effect immediately, but it’s also more cross platform. For example, instead of running the command above, we could have done this:
def commands(): import shutil import os.path path = os.path.expanduser("~/.foo_plugin") if os.path.exists(path): shutil.rmtree(path)
- comment(arg: str)#
Creates a comment line in the converted shell script code. This is only visible if the user views the current shell’s code using the command
rez-context --interpret
or looks at the file referenced by the environment variableREZ_CONTEXT_FILE
. You would create a comment for debugging purposes.if "nuke" in resolve: comment("note: taking over 'nuke' binary!") alias("nuke", "foo_nuke_replacer")
- defined(envvar: str) bool #
Use this boolean function to determine whether or not an environment variable is set.
if defined("REZ_MAYA_VERSION"): env.FOO_MAYA = 1
- env: dict#
The
env
object represents the environment dict of the configured environment. Environment variables can also be accessed as attributes.Note
Note that this is different from the standard python
os.environ
dict, which represents the current environment, not the one being configured. If a prior package’scommands()
sets a variable via theenv
object, it will be visible only viaenv
, notos.environ
. Theos.environ
dict hasn’t been updated because the target configured environment does not yet exist!env.FOO_DEBUG = 1 env["BAH_LICENSE"] = "/lic/bah.lic"
- env.append(value: str)#
Appends a value to an environment variable. By default this will use the
os.pathsep
delimiter between list items, but this can be overridden using the config settingenv_var_separators
. See Variable Appending And Prepending for further information on the behavior of this function.env.PATH.append("{root}/bin")
- env.prepend(value: str)#
Like
env.append()
, but prepends the environment variable instead.env.PYTHONPATH.prepend("{root}/python")
- ephemerals#
A dict like object representing the list of ephemerals in the resolved environment. Each item is a string (the full request, eg
.foo.cli-1
), keyed by the ephemeral package name. Note that you do not include the leading.
when getting items from theephemerals
object.Example:
if "foo.cli" in ephemerals: info("Foo cli option is being specified!")
- ephemerals.get_range(name: str, range_: str) VersionRange #
Use
get_range
to test with theintersects()
function. Here, we enablefoo
’s commandline tools by default, unless explicitly disabled via a request for.foo.cli-0
:if intersects(ephemerals.get_range("foo.cli", "1"), "1"): info("Enabling foo cli tools") env.PATH.append("{root}/bin")
- error(message: str)#
Prints to standard error.
Note
This function just prints the error, it does not prevent the target environment from being constructed (use the
stop()
) command for that).if "PyQt" in resolve: error("The floob package has problems running in combo with PyQt")
- expandable(arg: str) EscapedString #
- getenv(envvar: str)#
Gets the value of an environment variable.
if getenv("REZ_MAYA_VERSION") == "2016.sp1": pass
- Raises:
RexUndefinedVariableError – if the environment variable is not set.
- implicits#
A dict like object that is similar to the
request
object, but it contains only the package request as defined by theimplicit_packages
configuration setting.if "platform" in implicits: pass
- intersects(range1: str | VersionRange | VariantBinding | VersionBinding, range2: str) bool #
A boolean function that returns True if the version or version range of the given object, intersects with the given version range. Valid objects to query include:
A resolved package, eg
resolve.maya
;A package request, eg
request.foo
;A version of a resolved package, eg
resolve.maya.version
;A resolved ephemeral, eg
ephemerals.foo
;A version range object, eg
ephemerals.get_range('foo.cli', '1')
Warning
Do not do this:
if intersects(ephemerals.get("foo.cli", "0"), "1"): ...
If
foo.cli
is not present, this will unexpectedly compare the unversioned package named0
against the version range1
, which will succeed! Useephemerals.get_range()
andrequest.get_range
functions instead:if intersects(ephemerals.get_range("foo.cli", "0"), "1"): ...
Example:
if intersects(resolve.maya, "2019+"): info("Maya 2019 or greater is present")
- literal(arg: str) EscapedString #
Inhibits expansion of object and environment variable references.
env.FOO = literal("this {root} will not expand")
You can also chain together
literal
andexpandable()
functions like so:env.FOO = literal("the value of {root} is").expandable("{root}")
- optionvars(name: str, default: Any | None = None) Any #
A
dict.get()
like function for package accessing arbitrary data fromoptionvars
in rez config.
- request: RequirementsBinding#
A dict like object representing the list of package requests. Each item is a request string keyed by the package name. For example, consider the package request:
]$ rez-env maya-2015 maya_utils-1.2+<2 !corelib-1.4.4
This request would yield the following
request
object:{ "maya": "maya-2015", "maya_utils": "maya_utils-1.2+<2", "corelib": "!corelib-1.4.4" }
Use
get_range
to test with theintersects()
function:- if intersects(request.get_range(“maya”, “0”), “2019”):
info(“maya 2019.* was asked for!”)
Example:
if "maya" in request: info("maya was asked for!")
Tip
If multiple requests are present that refer to the same package, the request is combined ahead of time. In other words, if requests
foo-4+
andfoo-<6
were both present, the single requestfoo-4+<6
would be present in therequest
object.
- resolve#
A dict like object representing the list of packages in the resolved environment. Each item is a Package object, keyed by the package name.
Packages can be accessed using attributes (ie
resolve.maya
).if "maya" in resolve: info("Maya version is %s", resolve.maya.version) # ..or resolve["maya"].version
- setenv(envvar: str, value: str)#
This function sets an environment variable to the given value. It is equivalent to setting a variable via the
env
object (eg,env.FOO = 'BAH'
).setenv("FOO_PLUGIN_PATH", "{root}/plugins")
- source(path: str) None #
Source a shell script. Note that, similarly to
commands()
, this function cannot return a value, and any side effects that the script sourcing has is not visible to any packages. For example, if theinit.sh
script below containedexport FOO=BAH
, a subsequent test for this variable on theenv
object would yield nothing.source("{root}/scripts/init.sh")
- stop(message: str) NoReturn #
Raises an exception and stops a resolve from completing. You should use this when an unrecoverable error is detected and it is not possible to configure a valid environment.
stop("The value should be %s", expected_value)
- system: System#
This object provided system information, such as current platform, arch and os.
if system.platform == "windows": ...
- test#
Dict like object to access test related attributes. Only available in the
pre_test_commands()
function. Keys can be accessed as object attributes.
- test.name: str#
Name of the test about to run.
if test.name == "unit": info("My unit test is about to run yay")
- this#
The
this
object represents the current package. The following attributes are most commonly used in acommands()
) section (though you have access to all package attributes. See here):- this.base: str#
Similar to
this.root
, but does not include the variant subpath, if there is one. Different variants of the same package share the samebase
directory. See here for more information on package structure in relation to variants.
- this.root: str#
The installation directory of the package. If the package contains variants, this path will include the variant subpath. This is the directory that contains the installed package payload. See here for more information on package structure in relation to variants.
- this.version: VersionBinding#
The package version. It can be used as a string, however you can also access specific tokens in the version (such as major version number and so on), as this code snippet demonstrates:
env.FOO_MAJOR = this.version.major # or, this.version[0]
The available token references are
this.version.major
,this.version.minor
andthis.version.patch
, but you can also use a standard list index to reference any version token.
- undefined(envvar: str) bool #
Use this boolean function to determine whether or not an environment variable is set. This is the opposite of
defined()
.if undefined("REZ_MAYA_VERSION"): info("maya is not present")
- unsetenv(envvar: str) None #
Unsets an environment variable. This function does nothing if the environment variable was not set.
unsetenv("FOO_LIC_SERVER")
- version: VersionBinding#
See
this.version
.