Gaffer Wiki - User contributions [en]

Adding Presets to Global Context Variables

2024-07-23T21:07:32Z

Ericmehl:

Global context variables can be useful for ensuring a context variable is always accessible and set for the entire script. Unless it is overridden by a [https://www.gafferhq.org/documentation/1.4.8.0/Reference/NodeReference/Gaffer/ContextVariables.html ContextVariables] or [https://www.gafferhq.org/documentation/1.4.8.0/Reference/NodeReference/Gaffer/ContextVariableTweaks.html ContextVariablesTweaks] node in the node graph, all nodes will have access to the same global context variable. There are some global context variables added by Gaffer by default when creating a new script. You can see these, and add your own, from the <code>Variables</code> tab in the <code>Settings</code> dialog accessed from the <code>File.Settings</code>menu item.

The default user interface for a global context variable can be useful for many situations, but like all plugs in Gaffer, it is possible to customize how the plugs for global context variables are presented to the user. One such customization is adding presets for the user to choose from. This ensures only valid values can be assigned to the context variable. This guide will describe how to add presets to the widget controlling a global context variable.

=== Adding Presets to the <code>Settings</code> Dialog ===
Presets can be added to plugs in the <code>Settings</code> dialog by assigning metadata to the value plug for a global context variable. There are three steps to adding the needed metadata.

==== Identify the target plug ====
Before we assign metadata, we need to identify which plug represents the context variable you want to add presets to. Global context variables are stored in the <code>variables</code> child of the script node. In the Python editor, these are accessed by <code>root["variables"]</code>.

The entries in <code>root["variables"]</code> are of the type <code>NameValuePlug</code>. These plugs themselves have three child plugs: <code>name</code>, <code>enabled</code> and <code>value</code>. The <code>name</code> plug sets the name of the context variable and the <code>value</code> sets the value of the variable. The <code>enabled</code> plug is used to enable or disable the variable. Note that the value of <code>name</code> may be different from the name of the actual <code>NameValuePlug</code>.

To determine which child of <code>root["variables"]</code> you want to add metadata to, loop through all the children of <code>root["variables"]</code> checking the value of the <code>name</code> plug until you find the variable. The corresponding <code>value</code> child plug will be the plug we add metadata to.

==== Set the widget type ====
Next we need to tell Gaffer to use a presets dropdown widget for our value plug. That is done by setting <code>plugValueWidget:type</code> to <code>"GafferUI.PresetsPlugValueWidget"</code> : <syntaxhighlight lang="python">
Gaffer.Metadata.registerValue( variablePlug, "plugValueWidget:type", "GafferUI.PresetsPlugValueWidget" )
</syntaxhighlight>

==== Set the preset names and values ====
Now we add the presets themselves. This can be done in one of two ways. You can add a metadata entry starting with <code>preset:</code> followed by the display string of your preset set to the preset value. For example, <code>Gaffer.Metadata.registerValue( variablePlug, "preset:Shot 1004", "shot1004")</code>will create a preset showing in the UI as <code>Shot 1004</code>. When the user chooses that preset, the actual value of the context variable will be <code>shot1004</code>.

The second method sets all of the preset names and all of the values in one step each. This can be done by setting the <code>presetNames</code> metadata entry to a Cortex string vector (<code>IECore.StringVectorData</code>)holding all of the preset names. Setting <code>presetValues</code> to a Cortex vector of some type, such as <code>IECoreStringVectorData</code> or <code>IECoreFloatVectorData</code>.<syntaxhighlight lang="python">
Gaffer.Metadata.registerValue( variablePlug, "presetNames", IECore.StringVectorData( presetNames ) )
Gaffer.Metadata.registerValue( variablePlug, "presetValues", IECore.StringVectorData( presetValues ) )
</syntaxhighlight>

==== Resetting the <code>Settings</code> dialog ====

Note that the <code>Settings</code> UI is built once and then reused for subsequent showings. This means after adding presets, you need to force Gaffer to refresh the <code>Settings</code> dialog. You can force Gaffer to rebuild the <code>Settings</code> dialog by running the following code in the Python editor:<syntaxhighlight lang="python">
sw = GafferUI.ScriptWindow.acquire(root)
for window in sw.childWindows():
if hasattr( window, "_settingsEditor" ) :
sw.removeChild(window)
</syntaxhighlight>

Once you have registered the metadata for a plug in this way, it will be saved with the script so there is no need to register the metadata again.

==== Example Script ====

The following script is an example of all of the above steps condensed into a reusable function for adding presets for a global context variable.
<syntaxhighlight lang="python">
# Sets the preset names and values for a global context variable.
# `scriptNode` is the script node which you want to set global context variables for.
# `variableName` is the name of the variable to set presets for.
# `presetNames` is a list of strings to set the preset names to.
# `presetValues` is an `IECore.*VectorData` holding a list of preset values
# for each corresponding item in `presetNames`
def globalContextPresets( scriptNode, variableName, presetNames, presetValues ) :
if len( presetNames ) != len( presetValues ) :
raise ValueError( "`presetNames` and `presetValues` must be the same length" )

# Find the value plug for `variableName` global context variable.
variablePlug = None
for p in scriptNode["variables"] :
if p["name"].getValue() == variableName :
variablePlug = p["value"]
break

if variablePlug is None :
raise RuntimeError( "\"{}\" not found in global context variables".format( variableName ) )

# Set the widget to a presets dropdown.
Gaffer.Metadata.registerValue( variablePlug, "plugValueWidget:type", "GafferUI.PresetsPlugValueWidget" )

# Register the names and values.
Gaffer.Metadata.registerValue( variablePlug, "presetNames", IECore.StringVectorData( presetNames ) )
Gaffer.Metadata.registerValue( variablePlug, "presetValues", presetValues )

# Set the value to the first preset.
variablePlug.setValue( presetValues[0] )

# Tell Gaffer to rebuild the `Settings` dialog.
sw = GafferUI.ScriptWindow.acquire( scriptNode )
for window in sw.childWindows():
if hasattr( window, "_settingsEditor" ) :
sw.removeChild(window)
</syntaxhighlight>For example, to set a few presets for a global context variable named <code>shot</code> run the script above in the Python editor, then run the line below.<syntaxhighlight lang="python">
globalContextPresets( root, "shot", ["shot A", "shot B", "shot C"], IECore.StringVectorData( [ "shotA", "shotB", "shotC" ] ) )
</syntaxhighlight>

=== Adding Presets to a Menu ===
Global context variables can also be controlled from a customized menu in Gaffer's main application menu. In a script in your <code>~/gaffer/startup/gui</code> directory add the following code. It will add new menu item to Gaffer called <code>Shot</code>. Each time the user opens it, it will be rebuilt by the method <code>__shotMenu()</code>, which you can customize to populate the shot menu with shot names from any source accessible to Python such as a node in the node graph or an external database.
<syntaxhighlight lang="python">
import functools

import IECore

import GafferUI

# This will be called each time the user makes a selection from the `Shot` menu.
def __setShot( shotPlug, shot, checked ) :
if shotPlug is not None :
shotPlug.setValue( shot )

# This will be called for each menu item when showing the menu. It should return `True`
# if it should be checked and `False` if it should not be.
def __activeShot( shotPlug, shot ) :
if shotPlug is not None :
return shotPlug.getValue() == shot

# This will be called each time the user opens the `Shot` menu. It should return an
# `IECore.MenuDefinition` object representing the menu.
def __shotMenu( menu ) :
globalContextVariable = "shot"
# Populate the menu with placeholder item if no plug is found for `globalContextVariable`
defaultItems = [ "None" ]

scriptWindow = menu.ancestor(GafferUI.ScriptWindow)
currentScript = scriptWindow.scriptNode()

# Find the plug for the value of the global context variable.
shotPlug = None
for p in currentScript["variables"] :
if p["name"].getValue() == globalContextVariable :
shotPlug = p["value"]

result = IECore.MenuDefinition()

if shotPlug is None :
shotList = defaultItems
else :
# Hardcoded shot list
shotList = ["shotA", "shotB", "shotC"]

# Shot list from the enabled rows of a spreadsheet named "shots"
# shotList = currentScript["shots"]["enabledRowNames"].getValue()

# Add a shot for each item in `shotList`. When selecting the item, the `__setShot()`
# function will be called with the the `shotPlug`, the `shot` value from the loop
# and a `checked` variable the Gaffer menu system adds.
# When determining if the item should have a checkmark next to it, `__activeShot()` will
# be called with the `shotPlug` and `shot` value from the loop.
for shot in shotList :
result.append(
"/" + shot,
{
"command": functools.partial( __setShot, shotPlug, shot ) ,
"checkBox": functools.partial( __activeShot, shotPlug, shot ),
}
)

return result

# Called once at the start of a Gaffer session. Note that this will not be called
# on subsequent loading of a script, since the newly created window will be in the same session.
scriptWindowMenu = GafferUI.ScriptWindow.menuDefinition( application )
scriptWindowMenu.append( "/Shot", { "subMenu": __shotMenu } )
</syntaxhighlight>

Adding Presets to Global Context Variables

2024-07-23T21:00:55Z

Ericmehl:

Global context variables can be useful for ensuring a context variable is always accessible and set for the entire script. Unless it is overridden by a [https://www.gafferhq.org/documentation/1.4.8.0/Reference/NodeReference/Gaffer/ContextVariables.html ContextVariables] or [https://www.gafferhq.org/documentation/1.4.8.0/Reference/NodeReference/Gaffer/ContextVariableTweaks.html ContextVariablesTweaks] node in the node graph, all nodes will have access to the same global context variable. There are some global context variables added by Gaffer by default when creating a new script. You can see these, and add your own, from the <code>Variables</code> tab in the <code>Settings</code> dialog accessed from the <code>File.Settings</code>menu item.

The default user interface for a global context variable can be useful for many situations, but like all plugs in Gaffer, it is possible to customize how the plugs for global context variables are presented to the user. One such customization is adding presets for the user to choose from. This ensures only valid values can be assigned to the context variable. This guide will describe how to add presets to the widget controlling a global context variable.

=== Adding Presets to the <code>Settings</code> Dialog ===
Presets can be added to plugs in the <code>Settings</code> dialog by assigning metadata to the value plug for a global context variable. There are three steps to adding the needed metadata.

==== Identify the target plug ====
Before we assign metadata, we need to identify which plug represents the context variable you want to add presets to. Global context variables are stored in the <code>variables</code> child of the script node. In the Python editor, these are accessed by <code>root["variables"]</code>.

The entries in <code>root["variables"]</code> are of the type <code>NameValuePlug</code>. These plugs themselves have three child plugs: <code>name</code>, <code>enabled</code> and <code>value</code>. The <code>name</code> plug sets the name of the context variable and the <code>value</code> sets the value of the variable. The <code>enabled</code> plug is used to enable or disable the variable. Note that the value of <code>name</code> may be different from the name of the actual <code>NameValuePlug</code>.

To determine which child of <code>root["variables"]</code> you want to add metadata to, loop through all the children of <code>root["variables"]</code> checking the value of the <code>name</code> plug until you find the variable. The corresponding <code>value</code> child plug will be the plug we add metadata to.

==== Set the widget type ====
Next we need to tell Gaffer to use a presets dropdown widget for our value plug. That is done by setting <code>plugValueWidget:type</code> to <code>"GafferUI.PresetsPlugValueWidget"</code> : <syntaxhighlight lang="python">
Gaffer.Metadata.registerValue( variablePlug, "plugValueWidget:type", "GafferUI.PresetsPlugValueWidget" )
</syntaxhighlight>

==== Set the preset names and values ====
Now we add the presets themselves. This can be done in one of two ways. You can add a metadata entry starting with <code>preset:</code> followed by the display string of your preset set to the preset value. For example, <code>Gaffer.Metadata.registerValue( variablePlug, "preset:Shot 1004", "shot1004")</code>will create a preset showing in the UI as <code>Shot 1004</code>. When the user chooses that preset, the actual value of the context variable will be <code>shot1004</code>.

The second method sets all of the preset names and all of the values in one step each. This can be done by setting the <code>presetNames</code> metadata entry to a Cortex string vector (<code>IECore.StringVectorData</code>)holding all of the preset names. Setting <code>presetValues</code> to a Cortex vector of some type, such as <code>IECoreStringVectorData</code> or <code>IECoreFloatVectorData</code>.<syntaxhighlight lang="python">
Gaffer.Metadata.registerValue( variablePlug, "presetNames", IECore.StringVectorData( presetNames ) )
Gaffer.Metadata.registerValue( variablePlug, "presetValues", IECore.StringVectorData( presetValues ) )
</syntaxhighlight>

==== Resetting the <code>Settings</code> dialog ====

Note that the <code>Settings</code> UI is built once and then reused for subsequent showings. This means after adding presets, you need to force Gaffer to refresh the <code>Settings</code> dialog. You can force Gaffer to rebuild the <code>Settings</code> dialog by running the following code in the Python editor:<syntaxhighlight lang="python">
sw = GafferUI.ScriptWindow.acquire(root)
for window in sw.childWindows():
if hasattr( window, "_settingsEditor" ) :
sw.removeChild(window)
</syntaxhighlight>

Once you have registered the metadata for a plug in this way, it will be saved with the script so there is no need to register the metadata again.

==== Example Script ====

The following script is an example of all of the above steps condensed into a reusable function for adding presets for a global context variable.
<syntaxhighlight lang="python">
# Sets the preset names and values for a global context variable.
# `scriptNode` is the script node which you want to set global context variables for.
# `variableName` is the name of the variable to set presets for.
# `presetNames` is a list of strings to set the preset names to.
# `presetValues` is an `IECore.*VectorData` holding a list of preset values
# for each corresponding item in `presetNames`
def globalContextPresets( scriptNode, variableName, presetNames, presetValues ) :
if len( presetNames ) != len( presetValues ) :
raise ValueError( "`presetNames` and `presetValues` must be the same length" )

# Find the value plug for `variableName` global context variable.
variablePlug = None
for p in scriptNode["variables"] :
if p["name"].getValue() == variableName :
variablePlug = p["value"]
break

if variablePlug is None :
raise RuntimeError( "\"{}\" not found in global context variables".format( variableName ) )

# Set the widget to a presets dropdown.
Gaffer.Metadata.registerValue( variablePlug, "plugValueWidget:type", "GafferUI.PresetsPlugValueWidget" )

# Register the names and values.
Gaffer.Metadata.registerValue( variablePlug, "presetNames", IECore.StringVectorData( presetNames ) )
Gaffer.Metadata.registerValue( variablePlug, "presetValues", presetValues )

# Set the value to the first preset.
variablePlug.setValue( presetValues[0] )

# Tell Gaffer to rebuild the `Settings` dialog.
sw = GafferUI.ScriptWindow.acquire( scriptNode )
for window in sw.childWindows():
if hasattr( window, "_settingsEditor" ) :
sw.removeChild(window)
</syntaxhighlight>For example, to set a few presets for a global context variable named <code>shot</code> run the script above in the Python editor, then run the line below.<syntaxhighlight lang="python">
globalContextPresets( root, "shot", ["shot A", "shot B", "shot C"], IECore.StringVectorData( [ "shotA", "shotB", "shotC" ] ) )
</syntaxhighlight>

=== Adding Presets to a Menu ===
Global context variables can also be controlled from a customized menu in Gaffer's main application menu. In a script in your <code>~/gaffer/startup/gui</code> directory add the following code. It will add new menu item to Gaffer called <code>Shot</code>. Each time the user opens it, it will be rebuilt by the method `__shotMenu()`, which you can customize to populate the shot menu with shot names from any source accessible to Python such as a node in the node graph or an external database.<pre>
import functools

import IECore

import GafferUI

# This will be called each time the user makes a selection from the `Shot` menu.
def __setShot( shotPlug, shot, checked ) :
if shotPlug is not None :
shotPlug.setValue( shot )

# This will be called for each menu item when showing the menu. It should return `True`
# if it should be checked and `False` if it should not be.
def __activeShot( shotPlug, shot ) :
if shotPlug is not None :
return shotPlug.getValue() == shot

# This will be called each time the user opens the `Shot` menu. It should return an
# `IECore.MenuDefinition` object representing the menu.
def __shotMenu( menu ) :
globalContextVariable = "shot"
# Populate the menu with placeholder item if no plug is found for `globalContextVariable`
defaultItems = [ "None" ]

scriptWindow = menu.ancestor(GafferUI.ScriptWindow)
currentScript = scriptWindow.scriptNode()

# Find the plug for the value of the global context variable.
shotPlug = None
for p in currentScript["variables"] :
if p["name"].getValue() == globalContextVariable :
shotPlug = p["value"]

result = IECore.MenuDefinition()

if shotPlug is None :
shotList = defaultItems
else :
# Hardcoded shot list
shotList = ["shotA", "shotB", "shotC"]

# Shot list from the enabled rows of a spreadsheet named "shots"
# shotList = currentScript["shots"]["enabledRowNames"].getValue()

# Add a shot for each item in `shotList`. When selecting the item, the `__setShot()`
# function will be called with the the `shotPlug`, the `shot` value from the loop
# and a `checked` variable the Gaffer menu system adds.
# When determining if the item should have a checkmark next to it, `__activeShot()` will
# be called with the `shotPlug` and `shot` value from the loop.
for shot in shotList :
result.append(
"/" + shot,
{
"command": functools.partial( __setShot, shotPlug, shot ) ,
"checkBox": functools.partial( __activeShot, shotPlug, shot ),
}
)

return result

# Called once at the start of a Gaffer session. Note that this will not be called
# on subsequent loading of a script, since the newly created window will be in the same session.
scriptWindowMenu = GafferUI.ScriptWindow.menuDefinition( application )
scriptWindowMenu.append( "/Shot", { "subMenu": __shotMenu } )
</pre>

Adding Presets to Global Context Variables

2024-07-19T20:35:14Z

Ericmehl:

Adding Presets to Global Context Variables

2024-07-19T20:19:13Z

Ericmehl:

Adding Presets to Global Context Variables

2024-07-19T20:01:51Z

Ericmehl:

Adding Presets to Global Context Variables

2024-07-16T21:34:07Z

Ericmehl: Created page with "Global context variables can be useful for ensuring a context variable is always accessible and set for the entire script. Unless it is overridden by a [https://www.gafferhq.org/documentation/1.4.8.0/Reference/NodeReference/Gaffer/ContextVariables.html ContextVariables] or [https://www.gafferhq.org/documentation/1.4.8.0/Reference/NodeReference/Gaffer/ContextVariableTweaks.html ContextVariablesTweaks] node, all nodes will see the global context variable. There are some g..."

Main Page

2024-07-05T14:12:15Z

Ericmehl: Added guide for global context variable presets

{{DISPLAYTITLE:Gaffer Wiki}}
=Welcome to Gaffer Wiki=

Gaffer Wiki is a community effort and everyone is welcome to contribute. Here you can find [[:Category:Examples|Examples]], Templates, Guides and more for [https://www.gafferhq.org Gaffer]. See [[Help:Getting started|Getting started]] if this is your first time contributing.

== Getting Started With Gaffer ==

If you're completely new to Gaffer, begin with [[Getting started with Gaffer]].

== Examples ==

[[:Category:Examples|Examples]] provide small snippets of functionality to demonstrate how to do useful things with Gaffer's [[:Category:Nodes|Nodes]].

== Templates ==

Templates are more complete scripts, useful as a jumping off point for creating your own Gaffer setups.

== Guides ==

Guides go more in depth into specific aspects of Gaffer. From usage through to customising and extending Gaffer.

* [[Shortcuts and UI Interactions]]
* [[Installing Gaffer on AWS with EC2 Image Builder]]
* [[Adding Presets to Global Context Variables]]

== External Resources ==

[https://www.gafferhq.org/documentation Gaffer's Documentation]

[https://github.com/gafferHQ/gaffer Gaffer GitHub]

[https://www.gafferhq.org/resources GafferHQ Resources]

[https://groups.google.com/g/gaffer-dev Gaffer-Dev Mailing List]

[https://discord.gg/sEm8dDw Gaffer Users Discord]

__NOTOC__

Installing Gaffer on AWS with EC2 Image Builder

2024-06-20T21:24:58Z

Ericmehl:

This is a guide to setting up Amazon Web Services EC2 Image Builder with Gaffer. The EC2 Image Builder is a service from AWS that allows you to create an '''AMI''' (Amazon Machine Image) for use with EC2 with all the software and configuration needed to run jobs on AWS. An '''EC2 instance''' is a preset configuration of processor, memory and network resources made available by AWS - the virtual equivalent of hardware. An EC2 instance needs to be launched with an AMI that defines the virtual machine that will be running - the operating system and installed software.

This guide is intended for users with some AWS knowledge but not expert level. After completing this guide, you will have a Linux-based EC2 AMI available to your AWS users that can be run on any EC2 instance type for running Gaffer tasks, optionally including Arnold rendering. These instances can be launched manually through the AWS management console, via Python scripts or the AWS terminal, or from applications like Thinkbox's Deadline.

=== Overview ===
The EC2 Image Builder is split into a number of different parts. We need to work with three of those to create our Gaffer image. [https://docs.aws.amazon.com/imagebuilder/latest/userguide/start-build-image-pipeline.html '''Image Pipelines'''] are run to create the AMI. With Image Pipelines, you determine the schedule with which it will be run (likely Manual for our purposes) and the Image Recipe that will be used to create the AMI.

An '''[https://docs.aws.amazon.com/imagebuilder/latest/userguide/manage-recipes.html Image Recipe]''' consists the the base image AMI that will be the starting point for the new AMI, optional additional storage to attach to the instance and a set of components that define the software to be installed.

A '''[https://docs.aws.amazon.com/imagebuilder/latest/userguide/manage-components.html Component]''' is a YAML script describing the set of steps to be performed to install and configure some logical unit of software.

Image Recipes and Components are both versioned. When you save, that version becomes immutable and you will create a new version when making changes or fixes.

This guide will show you how to create Components that can be used to install any version of Gaffer and Arnold through the use of Component parameters. This means you can only have one version of Gaffer and Arnold installed on your AMI. Alternatively, you could create a new Component for each version of Gaffer / Arnold then choose multiple components to install on the same AMI.

=== Getting Started ===
To start, log into the AWS Console. Pay attention to the region you have set - this setting is in the top-right of the console. The EC2 Image Builder components as well as the generated AMIs (and therefore the EC2 instances you launch with them) are unique per-region.

Choose EC2 Image Builder from the Services menu in the top-left. You can navigate between Image Pipelines, Image Recipes and Components from the left-side menu.

=== Gaffer Component ===
From the navigation menu, choose Components to see a list of your customized components, which is empty if this is your first time using it.

# Choose "Create component".
# Keep "Component type" set to <code>Build</code>
# Set the Component Details
## Choose <code>Linux</code> as the "Image operating system". This indicates this recipe is compatible with Linux-based Image Pipelines. You determine the operating system for your AMI in the Image Recipe configuration.
## From "Compatible OS Versions", choose the Linux versions you intend for this recipe to be compatible with.
## Enter a Component name and optionally a description.
## Enter a Component version, such as <code>1.0.0</code>
# Set the Content. This is the YAML script that will be run to install Gaffer. The YAML script below will setup Gaffer.
# Finally, choose "Save Component" at the bottom of the screen to save it.

<pre>
name: Gaffer Component
description: Installs software needed for running Gaffer on AWS.
schemaVersion: 1.0

parameters:
- gafferVersion:
type: string
default: '1.4.7.0'
description: 'The Gaffer version to install.'
- gccVersion:
type: string
default: '9'
description: 'The gcc version to target for the Gaffer install.'

phases:
- name: build
steps:
- name: DownloadGaffer
action: WebDownload
inputs:
- source: https://github.com/GafferHQ/gaffer/releases/download/{{gafferVersion}}/gaffer-{{gafferVersion}}-linux-gcc{{gccVersion}}.tar.gz
destination: /opt/gaffer-{{gafferVersion}}/gaffer-{{gafferVersion}}-gcc{{gccVersion}}.tar.gz

- name: ExtractGaffer
action: ExecuteBash
inputs:
commands:
- tar xvzf {{build.DownloadGaffer.inputs[0].destination}} --strip-components=1 -C /opt/gaffer-{{gafferVersion}}
- rm {{build.DownloadGaffer.inputs[0].destination}}
- echo 'GAFFER_ROOT=/opt/gaffer-{{gafferVersion}}' >> .env
- echo 'GAFFER_VERSION={{gafferVersion}}' >> .env

- name: validate
steps:
- name: ValidateGaffer
action: ExecuteBash
inputs:
commands:
- source .env
- $GAFFER_ROOT/bin/gaffer env python -c "print('Gaffer validated')"

</pre>The <code>name</code>, <code>description</code>, <code>schemaVersion</code> and the two items under <code>phases</code> are required. The <code>parameters</code> list is a helpful way of specifying variables. They can be referenced in multiple places, so they make your Components less error-prone. Their value can also be set from within an Image Recipe which makes for much fewer versions of your Component. Without <code>parameter</code> lists, you would need to version up your Component with each Gaffer version.

==== Build Phase ====
The first phase of a component build installs Gaffer. It first downloads Gaffer from the Github release URL. Then it extracts the archive to the <code>/opt</code> directory. Finally, an environment variable is set for other components to locate Gaffer's root directory.

The extraction step makes use of another helpful feature in Component scripts - referencing variables from a previous step. The <code><nowiki>{{build.DownloadGaffer.inputs[0].destination}}</nowiki></code> text indicates that the file path to extract is taken from the <code>DownloadGaffer</code> step of the <code>build</code> phase. This is another way to reduce errors from typos in your script.

Note that starting with Gaffer 1.4, a gcc11 and gcc9 build is available. Which of these archives you use depends on the Linux distribution you use as your base image (see "Choosing a base image below" for more information). If you're running on Amazon Linux 2, you should use the gcc9 build as the sample above does. If you're using the newer Amazon Linux 2023, you should use the gcc11 build.

==== Validate Phase ====
The second phase is for verifying that your installation succeeded. For our purposes, running a short Python script from within Gaffer will prove that the installation succeeded and Gaffer is operable. Implementing this phase helps identify problems before the AMI is created. If any of the validation steps fail, the Image Pipeline will be stopped and an error raised.

=== Arnold Component ===
Creating the Arnold Component is much like the Gaffer Component, with one significant difference. Arnold's installer is not available publicly so you will need to upload it to a location that EC2 Image Builder can access. It should be kept private to your organization, which makes AWS S3 a good solution. '''S3''' is a storage repository you can manually or programmatically upload to. EC2 Image Builder Components have an action for downloading from S3, which makes it easy to incorporate into Components.

# Upload the Arnold Linux archive to a bucket in your S3 storage and note it's URL. It will start with <code>s3://</code>.
# Choose "Create Component" to start a new Component.
# Set the Component details as you did with the Gaffer Component.
# An example Component Content script is below.
<pre>
name: Arnold Component
description: Installs Arnold for AWS.
schemaVersion: 1.0

parameters:
- arnoldVersion:
type: string
default: '7.2.5.3'
description: 'The Arnold version to install.'

phases:
- name: build
steps:
- name: DownloadArnold
action: S3Download
inputs:
- source: s3://hypotheticalinc-image-sources/Arnold-{{arnoldVersion}}-linux.tgz
destination: /opt/arnoldRoot/arnold-{{arnoldVersion}}.tgz

- name: ExtractArnold
action: ExecuteBash
inputs:
commands:
- mkdir /opt/arnoldRoot/{{arnoldVersion}}
- tar xvzf {{build.DownloadArnold.inputs[0].destination}} -C /opt/arnoldRoot/{{arnoldVersion}}
- rm {{build.DownloadArnold.inputs[0].destination}}
- echo 'export ARNOLD_ROOT=/opt/arnoldRoot/{{arnoldVersion}}' >> .env

- name: validate
steps:
- name: ValidateGafferArnold
action: ExecuteBash
inputs:
commands:
- source .env
- $GAFFER_ROOT/bin/gaffer env python -c "import GafferArnold;GafferArnold.ArnoldLight()"

</pre>The Arnold component is similar to the Gaffer Component. Instead of downloading from a publicly available Github release, we get the Arnold archive from our S3 upload. This method can be used for other private archives you may need to install, such as non-public Gaffer extensions.

The validation phase once again runs a Python script in Gaffer. Here we simply import <code>GafferArnold</code> and attempt to create an <code>ArnoldLight</code> to verify it is working.

=== Gaffer Prerequisites Component ===
Depending on the which Linux distribution you are using as the base image for your Image Pipeline, you may need some additional packages installed to run Gaffer. The Amazon Linux 2 base image, for example, needs <code>mesa-libGL</code> and <code>mesa-libGLU</code> to run Gaffer tasks. These can be installed with the following Component.<pre>
name: Gaffer Prerequisites
description: Required components to run Gaffer.
schemaVersion: 1.0

phases:
- name: build
steps:
- name: installPackages
action: ExecuteBash
inputs:
commands:
- yum install -y mesa-libGL mesa-libGLU
</pre>

=== Image Recipe ===
With your Components defined, you can now create the Image Recipe for generating your AMI. Choose "Image Recipes". The Recipe list will be empty if you haven't created on yet.

# Choose "Create image recipe".
# Enter a name for your Recipe and optionally a description.
# Enter a version number, such as <code>1.0.0</code>.
# Select an AMI to use as the base image for the Recipe. See the section below for more information about choosing a base image.
# Enable your <code>Gaffer</code>, <code>Arnold</code> and <code>GafferPrerequisites</code> Components you created above. You may need to change the search filter to <code>Owned by me</code>.
# As you enable each Component, it will be added to the "Selected Components" section.
# The Components will be added in the order they are listed. Make sure the <code>GafferPrerequisites</code> Component is first, followed by <code>Gaffer</code> and then <code>Arnold</code>. If they are not in the correct order, validation will fail.
# You can modify the version number to install in the "Selected Components" section. This is also where you can change the installed versions when making a new version of this Image Recipe for future releases.
# It is also a good idea to change the "Versioning Options" of your Components to <code>Use latest available component version</code>. This will ensure the latest component is used without needing to create new Image Recipe versions each time a Component is modified. Using the version variables in your Component allows you to update new software versions without needing to change the Component, only the Recipe.
# If you need to attach additional storage, you can do that in the "Storage (volumes)" section. This can be useful for storing assets on the instance.
# Choose "Create Recipe" to complete the Recipe.

==== Choosing a base image ====
The base image is the AMI that will be used as the starting point for your customized AMI. It is booted first, your components are added to it, and a snapshot is taken to be used as your custom AMI.

The Image Recipe creation wizard allows you to choose a variety of standard Linux distributions, images you've subscribed to from the AWS Marketplace, or a manually entered AMI ID.

If you are using Deadline, entering an AMI ID can be helpful for identifying the AMI provided by AWS / Thinkbox that has the needed Deadline components preinstalled. One way of finding this ID is to launch a Deadline worker from the AWS Portal in the Deadline Monitor. There are various options for AMIs with DCC software preinstalled. You can choose the generic Linux worker for a clean base AMI. Once the machine has started, go to your EC2 Instances in the AWS Web Console, locate the running instance and note its AMI ID.

=== Image Pipeline ===
The final step is to create an Image Pipeline. Choose Image Pipelines from the navigation menu to see the list of Pipelines, which is empty if this is your first time using it.

# Choose "Create Image Pipeline" to get started.
# Enter a name for the pipeline and optionally a description.
# Change the schedule option to "Manual" so it will only run when you trigger it.
# Proceed to the next step, to configure the Recipe.
# Choose the Recipe you created above.
# Proceed through the remaining wizard screens, accepting the default values.
When you are finished, refresh the Image Pipeline list. Select your new Pipeline and choose "Run Pipeline" from the "Actions" menu. Choosing the Pipeline in the list will take you to a list of Pipeline runs, your new submission should be in the <code>Building</code> status.

Once the image has been built and validated, it will be available in your EC2 AMI list for use!

=== Debugging ===
If your AMI is not successfully built, you can view the log stream for a detailed description of the steps EC2 Image Builder performed, including the results of your Component validation steps. This can be useful for finding what went wrong and making corrections to your Components.

=== Updating Installed Versions ===

# Select your Image Recipe.
# Choose "Create new version".
# Enter the new version numbers in the Component settings.
# Either create a new Image Pipeline for your new Image Recipe, or modify the existing Image Pipeline to use it in place.

=== GafferDeadline Component ===
Though it isn't necessary for using Gaffer on AWS, if you are using Deadline to queue jobs to AWS, you will also need to have GafferDeadline installed. The following script can be used as a Component for GafferDeadline.<pre>
name: GafferDeadline Component
description: Installs GafferDeadline for AWS.
schemaVersion: 1.0

parameters:
- gafferDeadlineVersion:
type: string
default: '0.58.0.0b2'
description: 'The GafferDeadline version to install.'

phases:
- name: build
steps:
- name: DownloadGafferDeadline
action: WebDownload
inputs:
- source: https://github.com/hypothetical-inc/GafferDeadline/archive/refs/tags/{{gafferDeadlineVersion}}.tar.gz
destination: /opt/GafferDeadline/gafferDeadline{{gafferDeadlineVersion}}.tar.gz

- name: ExtractGafferDeadline
action: ExecuteBash
inputs:
commands:
- tar -xvzf {{build.DownloadGafferDeadline.inputs[0].destination}} --strip-components=1 -C /opt/GafferDeadline
- rm {{build.DownloadGafferDeadline.inputs[0].destination}}

- name: validate
steps:
- name: ValidateGafferDeadline
action: ExecuteBash
inputs:
commands:
- source .env
- export GAFFER_EXTENSION_PATHS=/opt/GafferDeadline
- $GAFFER_ROOT/bin/gaffer env python -c "import GafferDeadline;print(help(GafferDeadline))"

</pre>

Installing Gaffer on AWS with EC2 Image Builder

2024-06-20T20:56:40Z

Ericmehl:

This is a guide to setting up Amazon Web Services EC2 Image Builder with Gaffer. The EC2 Image Builder is a service from AWS that allows you to create an '''AMI''' (Amazon Machine Image) for use with EC2 with all the software and configuration needed to run jobs on AWS. An '''EC2 instance''' is a preset configuration of processor, memory and network resources made available by AWS - the virtual equivalent of hardware. An EC2 instance needs to be launched with an AMI that defines the virtual machine that will be running - the operating system and installed software.

This guide is intended for users with some AWS knowledge but not expert level. After completing this guide, you will have a Linux-based EC2 AMI available to your AWS users that can be run on any EC2 instance type for running Gaffer tasks, optionally including Arnold rendering. These instances can be launched manually through the AWS management console, via Python scripts or the AWS terminal, or from applications like Thinkbox's Deadline.

=== Overview ===
The EC2 Image Builder is split into a number of different parts. We need to work with three of those to create our Gaffer image. [https://docs.aws.amazon.com/imagebuilder/latest/userguide/start-build-image-pipeline.html '''Image Pipelines'''] are run to create the AMI. With Image Pipelines, you determine the schedule with which it will be run (likely Manual for our purposes) and the Image Recipe that will be used to create the AMI.

An '''[https://docs.aws.amazon.com/imagebuilder/latest/userguide/manage-recipes.html Image Recipe]''' consists the the base image AMI that will be the starting point for the new AMI, optional additional storage to attach to the instance and a set of components that define the software to be installed.

A '''[https://docs.aws.amazon.com/imagebuilder/latest/userguide/manage-components.html Component]''' is a YAML script describing the set of steps to be performed to install and configure some logical unit of software.

Image Recipes and Components are both versioned. When you save, that version becomes immutable and you will create a new version when making changes or fixes.

This guide will show you how to create Components that can be used to install any version of Gaffer and Arnold through the use of Component parameters. This means you can only have one version of Gaffer and Arnold installed on your AMI. Alternatively, you could create a new Component for each version of Gaffer / Arnold then choose multiple components to install on the same AMI.

=== Getting Started ===
To start, log into the AWS Console. Pay attention to the region you have set - this setting is in the top-right of the console. The EC2 Image Builder components as well as the generated AMIs (and therefore the EC2 instances you launch with them) are unique per-region.

Choose EC2 Image Builder from the Services menu in the top-left. You can navigate between Image Pipelines, Image Recipes and Components from the left-side menu.

=== Gaffer Component ===
From the navigation menu, choose Components to see a list of your customized components, which is empty if this is your first time using it.

# Choose "Create component".
# Keep "Component type" set to <code>Build</code>
# Set the Component Details
## Choose <code>Linux</code> as the "Image operating system". This indicates this recipe is compatible with Linux-based Image Pipelines. You determine the operating system for your AMI in the Image Recipe configuration.
## From "Compatible OS Versions", choose the Linux versions you intend for this recipe to be compatible with.
## Enter a Component name and optionally a description.
## Enter a Component version, such as <code>1.0.0</code>
# Set the Content. This is the YAML script that will be run to install Gaffer. The YAML script below will setup Gaffer.
# Finally, choose "Save Component" at the bottom of the screen to save it.

<pre>
name: Gaffer Component
description: Installs Gaffer for running on AWS.
schemaVersion: 1.0

parameters:
- gafferVersion:
type: string
default: '1.3.7.0'
description: 'The Gaffer version to install.'

phases:
- name: build
steps:
- name: DownloadGaffer
action: WebDownload
inputs:
- source: https://github.com/GafferHQ/gaffer/releases/download/{{gafferVersion}}/gaffer-{{gafferVersion}}-linux.tar.gz
destination: /opt/gaffer-{{gafferVersion}}/gaffer-{{gafferVersion}}.tar.gz

- name: ExtractGaffer
action: ExecuteBash
inputs:
commands:
- tar xvzf {{build.DownloadGaffer.inputs[0].destination}} --strip-components=1 -C /opt/gaffer-{{gafferVersion}}
- rm {{build.DownloadGaffer.inputs[0].destination}}
- echo 'export GAFFER_ROOT=/opt/gaffer-{{gafferVersion}}' >> /etc/profile

- name: validate
steps:
- name: ValidateGaffer
action: ExecuteBash
inputs:
commands:
- $GAFFER_ROOT/bin/gaffer env python -c "print('Gaffer validated')"

</pre>The <code>name</code>, <code>description</code>, <code>schemaVersion</code> and the two items under <code>phases</code> are required. The <code>parameters</code> list is a helpful way of specifying variables. They can be referenced in multiple places, so they make your Components less error-prone. Their value can also be set from within an Image Recipe which makes for much fewer versions of your Component. Without <code>parameter</code> lists, you would need to version up your Component with each Gaffer version.

==== Build Phase ====
The first phase of a component build installs Gaffer. It first downloads Gaffer from the Github release URL. Then it extracts the archive to the <code>/opt</code> directory. Finally, an environment variable is set for other components to locate Gaffer's root directory.

The extraction step makes use of another helpful feature in Component scripts - referencing variables from a previous step. The <code><nowiki>{{build.DownloadGaffer.inputs[0].destination}}</nowiki></code> text indicates that the file path to extract is taken from the <code>DownloadGaffer</code> step of the <code>build</code> phase. This is another way to reduce errors from typos in your script.

Note that starting with Gaffer 1.4, a gcc11 and gcc9 build is available. Which of these archives you use depends on the Linux distribution you use as your base image (see "Choosing a base image below" for more information). If you're running on Amazon Linux 2, you should use the gcc9 build as the sample above does. If you're using the newer Amazon Linux 2023, you should use the gcc11 build.

==== Validate Phase ====
The second phase is for verifying that your installation succeeded. For our purposes, running a short Python script from within Gaffer will prove that the installation succeeded and Gaffer is operable. Implementing this phase helps identify problems before the AMI is created. If any of the validation steps fail, the Image Pipeline will be stopped and an error raised.

=== Arnold Component ===
Creating the Arnold Component is much like the Gaffer Component, with one significant difference. Arnold's installer is not available publicly so you will need to upload it to a location that EC2 Image Builder can access. It should be kept private to your organization, which makes AWS S3 a good solution. '''S3''' is a storage repository you can manually or programmatically upload to. EC2 Image Builder Components have an action for downloading from S3, which makes it easy to incorporate into Components.

# Upload the Arnold Linux archive to a bucket in your S3 storage and note it's URL. It will start with <code>s3://</code>.
# Choose "Create Component" to start a new Component.
# Set the Component details as you did with the Gaffer Component.
# An example Component Content script is below.
<pre>
name: Arnold Component
description: Installs Arnold and sets the ARNOLD_ROOT environment variable.
schemaVersion: 1.0

parameters:
- arnoldVersion:
type: string
default: '7.2.5.3'
description: 'The Arnold version to install.'

phases:
- name: build
steps:
- name: DownloadArnold
action: S3Download
inputs:
- source: s3://<YOUR BUCKET NAME>/Arnold-{{arnoldVersion}}-linux.tgz
destination: /opt/arnoldRoot/arnold-{{arnoldVersion}}.tgz

- name: ExtractArnold
action: ExecuteBash
inputs:
commands:
- mkdir /opt/arnoldRoot/{{arnoldVersion}}
- tar xvzf {{build.DownloadArnold.inputs[0].destination}} -C /opt/arnoldRoot/{{arnoldVersion}}
- rm {{build.DownloadArnold.inputs[0].destination}}
- echo 'export ARNOLD_ROOT=/opt/arnoldRoot/{{arnoldVersion}}' >> /etc/profile

- name: validate
steps:
- name: ValidateGafferArnold
action: ExecuteBash
inputs:
commands:
- $GAFFER_ROOT/bin/gaffer env python -c "import GafferArnold;print(help(GafferArnold))"

</pre>The Arnold component is similar to the Gaffer Component. Instead of downloading from a publicly available Github release, we get the Arnold archive from our S3 upload. This method can be used for other private archives you may need to install, such as non-public Gaffer extensions.

The validation phase once again runs a Python script in Gaffer. Here we simply import <code>GafferArnold</code> and attempt to create an <code>ArnoldLight</code> to verify it is working.

=== Gaffer Prerequisites Component ===
Depending on the which Linux distribution you are using as the base image for your Image Pipeline, you may need some additional packages installed to run Gaffer. The Amazon Linux 2 base image, for example, needs <code>mesa-libGL</code> and <code>mesa-libGLU</code> to run Gaffer tasks. These can be installed with the following Component.<pre>
name: Gaffer Prerequisites
description: Required components to run Gaffer.
schemaVersion: 1.0

phases:
- name: build
steps:
- name: installPackages
action: ExecuteBash
inputs:
commands:
- yum install -y mesa-libGL mesa-libGLU
</pre>

=== Image Recipe ===
With your Components defined, you can now create the Image Recipe for generating your AMI. Choose "Image Recipes". The Recipe list will be empty if you haven't created on yet.

# Choose "Create image recipe".
# Enter a name for your Recipe and optionally a description.
# Enter a version number, such as <code>1.0.0</code>.
# Select an AMI to use as the base image for the Recipe. See the section below for more information about choosing a base image.
# Enable your <code>Gaffer</code>, <code>Arnold</code> and <code>GafferPrerequisites</code> Components you created above. You may need to change the search filter to <code>Owned by me</code>.
# As you enable each Component, it will be added to the "Selected Components" section.
# The Components will be added in the order they are listed. Make sure the <code>GafferPrerequisites</code> Component is first, followed by <code>Gaffer</code> and then <code>Arnold</code>. If they are not in the correct order, validation will fail.
# You can modify the version number to install in the "Selected Components" section. This is also where you can change the installed versions when making a new version of this Image Recipe for future releases.
# It is also a good idea to change the "Versioning Options" of your Components to <code>Use latest available component version</code>. This will ensure the latest component is used without needing to create new Image Recipe versions each time a Component is modified. Using the version variables in your Component allows you to update new software versions without needing to change the Component, only the Recipe.
# If you need to attach additional storage, you can do that in the "Storage (volumes)" section. This can be useful for storing assets on the instance.
# Choose "Create Recipe" to complete the Recipe.

==== Choosing a base image ====
The base image is the AMI that will be used as the starting point for your customized AMI. It is booted first, your components are added to it, and a snapshot is taken to be used as your custom AMI.

The Image Recipe creation wizard allows you to choose a variety of standard Linux distributions, images you've subscribed to from the AWS Marketplace, or a manually entered AMI ID.

If you are using Deadline, entering an AMI ID can be helpful for identifying the AMI provided by AWS / Thinkbox that has the needed Deadline components preinstalled. One way of finding this ID is to launch a Deadline worker from the AWS Portal in the Deadline Monitor. There are various options for AMIs with DCC software preinstalled. You can choose the generic Linux worker for a clean base AMI. Once the machine has started, go to your EC2 Instances in the AWS Web Console, locate the running instance and note its AMI ID.

=== Image Pipeline ===
The final step is to create an Image Pipeline. Choose Image Pipelines from the navigation menu to see the list of Pipelines, which is empty if this is your first time using it.

# Choose "Create Image Pipeline" to get started.
# Enter a name for the pipeline and optionally a description.
# Change the schedule option to "Manual" so it will only run when you trigger it.
# Proceed to the next step, to configure the Recipe.
# Choose the Recipe you created above.
# Proceed through the remaining wizard screens, accepting the default values.
When you are finished, refresh the Image Pipeline list. Select your new Pipeline and choose "Run Pipeline" from the "Actions" menu. Choosing the Pipeline in the list will take you to a list of Pipeline runs, your new submission should be in the <code>Building</code> status.

Once the image has been built and validated, it will be available in your EC2 AMI list for use!

=== Debugging ===
If your AMI is not successfully built, you can view the log stream for a detailed description of the steps EC2 Image Builder performed, including the results of your Component validation steps. This can be useful for finding what went wrong and making corrections to your Components.

=== Updating Installed Versions ===

# Select your Image Recipe.
# Choose "Create new version".
# Enter the new version numbers in the Component settings.
# Either create a new Image Pipeline for your new Image Recipe, or modify the existing Image Pipeline to use it in place.

Installing Gaffer on AWS with EC2 Image Builder

2024-06-20T15:40:08Z

Ericmehl:

This is a guide to setting up Amazon Web Services EC2 Image Builder with Gaffer. The EC2 Image Builder is a service from AWS that allows you to create an '''AMI''' (Amazon Machine Image) for use with EC2 with all the software and configuration needed to run jobs on AWS. An '''EC2 instance''' is a preset configuration of processor, memory and network resources made available by AWS - the virtual equivalent of hardware. An EC2 instance needs to be launched with an AMI that defines the virtual machine that will be running - the operating system and installed software.

This guide is intended for users with some AWS knowledge but not expert level. After completing this guide, you will have a Linux-based EC2 AMI available to your AWS users that can be run on any EC2 instance type for running Gaffer tasks, including Arnold rendering. These instances can be launched manually through the AWS management console, via Python scripts or the AWS terminal, or from applications like Thinkbox's Deadline.

=== Overview ===
The EC2 Image Builder is split into a number of different parts. We need to work with three of those to create our Gaffer image. [https://docs.aws.amazon.com/imagebuilder/latest/userguide/start-build-image-pipeline.html '''Image Pipelines'''] are run to create the AMI. With Image Pipelines, you determine the schedule with which it will be run (likely Manual for our purposes) and the Image Recipe that will be used to create the AMI.

An '''[https://docs.aws.amazon.com/imagebuilder/latest/userguide/manage-recipes.html Image Recipe]''' consists the the base image AMI that will be the starting point for the new AMI, optional additional storage to attach to the instance and a set of components that define the software to be installed.

A '''[https://docs.aws.amazon.com/imagebuilder/latest/userguide/manage-components.html Component]''' is a YAML script describing the set of steps to be performed to install and configure some logical unit of software.

Image Recipes and Components are both versioned. When you save, that version becomes immutable and you will create a new version when making changes or fixes.

=== Getting Started ===
To start, log into the AWS Console. Pay attention to the region you have set - this setting is in the top-right of the console. The EC2 Image Builder components as well as the generated AMIs (and therefore the EC2 instances you launch with them) are unique per-region.

Choose EC2 Image Builder from the Services menu in the top-left. You can navigate between Image Pipelines, Image Recipes and Components from the left-side menu.

=== Gaffer Component ===
From the navigation menu, choose Components to see a list of your customized components, which is empty if this is your first time using it.

# Choose "Create component".
# Keep "Component type" set to <code>Build</code>
# Set the Component Details
## Choose <code>Linux</code> as the "Image operating system". This indicates this recipe is compatible with Linux-based Image Pipelines. You determine the operating system for your AMI in the Image Pipeline configuration.
## From "Compatible OS Versions", choose the Linux versions you intend for this recipe to be compatible with.
## Enter a Component name and optionally a description.
## Enter a Component version, such as <code>1.0.0</code>
# Set the Content. This is the YAML script that will be run to install Gaffer. The YAML script below will setup Gaffer.
# Finally, choose "Save Component" at the bottom of the screen to save it.

<pre>
name: Gaffer Component
description: Installs Gaffer for running on AWS.
schemaVersion: 1.0

parameters:
- gafferVersion:
type: string
default: '1.3.7.0'
description: 'The Gaffer version to install.'

phases:
- name: build
steps:
- name: DownloadGaffer
action: WebDownload
inputs:
- source: https://github.com/GafferHQ/gaffer/releases/download/{{gafferVersion}}/gaffer-{{gafferVersion}}-linux.tar.gz
destination: /opt/gaffer-{{gafferVersion}}/gaffer-{{gafferVersion}}.tar.gz

- name: ExtractGaffer
action: ExecuteBash
inputs:
commands:
- tar xvzf {{build.DownloadGaffer.inputs[0].destination}} --strip-components=1 -C /opt/gaffer-{{gafferVersion}}
- rm {{build.DownloadGaffer.inputs[0].destination}}
- echo 'export GAFFER_ROOT=/opt/gaffer-{{gafferVersion}}' >> /etc/profile

- name: validate
steps:
- name: ValidateGaffer
action: ExecuteBash
inputs:
commands:
- $GAFFER_ROOT/bin/gaffer env python -c "print('Gaffer validated')"

</pre>The <code>name</code>, <code>description</code>, <code>schemaVersion</code> and the two items under <code>phases</code> are required. The <code>parameters</code> list is a helpful way of specifying variables. They can be referenced in multiple places, so they make your Components less error-prone. Their value can also be set from within an Image Recipe which makes for much fewer versions of your Component. Without <code>parameter</code> lists, you would need to version up your Component with each Gaffer version.

==== Build Phase ====
The first phase of a component build installs Gaffer. It first downloads Gaffer from the Github release URL. Then it extracts the archive to the <code>/opt</code> directory. Finally, an environment variable is set for other components to locate Gaffer's root directory.

The extraction step makes use of another helpful feature in Component scripts - referencing variables from a previous step. The <code><nowiki>{{build.DownloadGaffer.inputs[0].destination}}</nowiki></code> text indicates that the file path to extract is taken from the <code>DownloadGaffer</code> step of the <code>build</code> phase. This is another way to reduce errors from typos in your script.

==== Validate Phase ====
The second phase is for verifying that your installation succeeded. For our purposes, running a short Python script from within Gaffer will prove that the installation succeeded and Gaffer is operable. Implementing this phase helps identify problems before the AMI is created. If any of the validation steps fail, the Image Pipeline will be stopped and an error raised.

=== Arnold Component ===
Creating the Arnold Component is much like the Gaffer Component, with one significant difference. Arnold's installer is not available publicly so you will need to upload it to a location that EC2 Image Builder can access. It should be kept private to your organization, which makes AWS S3 a good solution. '''S3''' is a storage repository you can manually or programmatically upload to. EC2 Image Builder Components have an action for downloading from S3, which makes it easy to incorporate into Components.

# Upload the Arnold Linux archive to a bucket in your S3 storage and note it's URL. It will start with <code>s3://</code>.
# Choose "Create Component" to start a new Component.
# Set the Component details as you did with the Gaffer Component.
# An example Component Content script is below.
<pre>
name: Arnold Component
description: Installs Arnold and sets the ARNOLD_ROOT environment variable.
schemaVersion: 1.0

parameters:
- arnoldVersion:
type: string
default: '7.2.5.3'
description: 'The Arnold version to install.'

phases:
- name: build
steps:
- name: DownloadArnold
action: S3Download
inputs:
- source: s3://<YOUR BUCKET NAME>/Arnold-{{arnoldVersion}}-linux.tgz
destination: /opt/arnoldRoot/arnold-{{arnoldVersion}}.tgz

- name: ExtractArnold
action: ExecuteBash
inputs:
commands:
- mkdir /opt/arnoldRoot/{{arnoldVersion}}
- tar xvzf {{build.DownloadArnold.inputs[0].destination}} -C /opt/arnoldRoot/{{arnoldVersion}}
- rm {{build.DownloadArnold.inputs[0].destination}}
- echo 'export ARNOLD_ROOT=/opt/arnoldRoot/{{arnoldVersion}}' >> /etc/profile

- name: validate
steps:
- name: ValidateGafferArnold
action: ExecuteBash
inputs:
commands:
- $GAFFER_ROOT/bin/gaffer env python -c "import GafferArnold;print(help(GafferArnold))"

</pre>The Arnold component is similar to the Gaffer Component. Instead of downloading from a publicly available Github release, we get the Arnold archive from our S3 upload. This method can be used for other private archives you may need to install, such as non-public Gaffer extensions.

The validation phase once again runs a Python script in Gaffer. Here we simply import <code>GafferArnold</code> and attempt to create an <code>ArnoldLight</code> to verify it is working.

=== Gaffer Prerequisites Component ===
Depending on the which Linux distribution you are using as the base image for your Image Pipeline, you may need some additional packages installed to run Gaffer. The Amazon Linux 2 base image, for example, needs <code>mesa-libGL</code> and <code>mesa-libGLU</code> to run Gaffer tasks. These can be installed with the following Component.<pre>
name: Gaffer Prerequisites
description: Required components to run Gaffer.
schemaVersion: 1.0

phases:
- name: build
steps:
- name: installPackages
action: ExecuteBash
inputs:
commands:
- yum install -y mesa-libGL mesa-libGLU
</pre>

=== Image Recipe ===
With your Components defined, you can now create the Image Recipe for generating your AMI. Choose "Image Recipes". The Recipe list will be empty if you haven't created on yet.

# Choose "Create image recipe".
# Enter a name for your Recipe and optionally a description.
# Enter a version number, such as <code>1.0.0</code>.
# Select an AMI to use as the base image for the Recipe. See the section below for more information about choosing a base image.
# Enable your <code>Gaffer</code>, <code>Arnold</code> and <code>GafferPrerequisites</code> Components you created above. You may need to change the search filter to <code>Owned by me</code>.
# As you enable each Component, it will be added to the "Selected Components" section.
# The Components will be added in the order they are listed. Make sure the <code>GafferPrerequisites</code> Component is first, followed by <code>Gaffer</code> and then <code>Arnold</code>. If they are not in the correct order, validation will fail.
# You can modify the version number to install in the "Selected Components" section. This is also where you can change the installed versions when making a new version of this Image Recipe for future releases.
# It is also a good idea to change the "Versioning Options" of your Components to <code>Use latest available component version</code>. This will ensure the latest component is used without needing to create new Image Recipe versions each time a Component is modified. Using the version variables in your Component allows you to update new software versions without needing to change the Component, only the Recipe.
# If you need to attach additional storage, you can do that in the "Storage (volumes)" section. This can be useful for storing assets on the instance.
# Choose "Create Recipe" to complete the Recipe.

==== Choosing a base image ====
The base image is the AMI that will be used as the starting point for your customized AMI. It is booted first, your components are added to it, and a snapshot is taken to be used as your custom AMI.

The Image Recipe creation wizard allows you to choose a variety of standard Linux distributions, images you've subscribed to from the AWS Marketplace, or a manually entered AMI ID.

If you are using Deadline, entering an AMI ID can be helpful for identifying the AMI provided by AWS / Thinkbox that has the needed Deadline components preinstalled. One way of finding this ID is to launch a Deadline worker from the AWS Portal in the Deadline Monitor. There are various options for AMIs with DCC software preinstalled. You can choose the generic Linux worker for a clean base AMI. Once the machine has started, go to your EC2 Instances in the AWS Web Console, locate the running instance and note its AMI ID.

=== Image Pipeline ===
The final step is to create an Image Pipeline. Choose Image Pipelines from the navigation menu to see the list of Pipelines, which is empty if this is your first time using it.

# Choose "Create Image Pipeline" to get started.
# Enter a name for the pipeline and optionally a description.
# Change the schedule option to "Manual" so it will only run when you trigger it.
# Proceed to the next step, to configure the Recipe.
# Choose the Recipe you created above.
# Proceed through the remaining wizard screens, accepting the default values.
When you are finished, refresh the Image Pipeline list. Select your new Pipeline and choose "Run Pipeline" from the "Actions" menu. Choosing the Pipeline in the list will take you to a list of Pipeline runs, your new submission should be in the <code>Building</code> status.

Once the image has been built and validated, it will be available in your EC2 AMI list for use!

=== Debugging ===
If your AMI is not successfully built, you can view the log stream for a detailed description of the steps EC2 Image Builder performed, including the results of your Component validation steps. This can be useful for finding what went wrong and making corrections to your Components.

=== Updating Installed Versions ===

# Select your Image Recipe.
# Choose "Create new version".
# Enter the new version numbers in the Component settings.
# Either create a new Image Pipeline for your new Image Recipe, or modify the existing Image Pipeline to use it in place.

Installing Gaffer on AWS with EC2 Image Builder

2024-06-19T21:36:22Z

Ericmehl:

Installing Gaffer on AWS with EC2 Image Builder

2024-06-19T21:09:46Z

Ericmehl:

Installing Gaffer on AWS with EC2 Image Builder

2024-06-19T20:27:43Z

Ericmehl:

Installing Gaffer on AWS with EC2 Image Builder

2024-06-19T20:11:50Z

Ericmehl:

Installing Gaffer on AWS with EC2 Image Builder

2024-06-19T20:06:42Z

Ericmehl:

Installing Gaffer on AWS with EC2 Image Builder

2024-06-19T19:21:36Z

Ericmehl: Created page with "This is a guide to setting up Amazon Web Services EC2 Image Builder with Gaffer. The EC2 Image Builder is a service from AWS that allows you to create AMIs (Amazon Machine Images) for use with EC2 with all the software and configuration needed to run jobs on AWS. This guide is intended for users with some AWS knowledge but not expert level. After completing this guide, you will have an EC2 AMI available to your AWS users that can be run on any EC2 instance type for runn..."

This is a guide to setting up Amazon Web Services EC2 Image Builder with Gaffer. The EC2 Image Builder is a service from AWS that allows you to create AMIs (Amazon Machine Images) for use with EC2 with all the software and configuration needed to run jobs on AWS.

This guide is intended for users with some AWS knowledge but not expert level. After completing this guide, you will have an EC2 AMI available to your AWS users that can be run on any EC2 instance type for running Gaffer tasks. These instances can be launched manually through the AWS management console, via Python scripts or AWS terminals, or from applications like Thinkbox's Deadline.

=== Overview ===
The EC2 Image Builder is split into number of different parts. We need to work with three of those to create our Gaffer image. Image Pipelines are what

Main Page

2024-06-19T18:55:54Z

Ericmehl:

{{DISPLAYTITLE:Gaffer Wiki}}
=Welcome to Gaffer Wiki=

Gaffer Wiki is a community effort and everyone is welcome to contribute. Here you can find [[:Category:Examples|Examples]], Templates, Guides and more for [https://www.gafferhq.org Gaffer]. See [[Help:Getting started|Getting started]] if this is your first time contributing.

== Getting Started With Gaffer ==

If you're completely new to Gaffer, begin with [[Getting started with Gaffer]].

== Examples ==

[[:Category:Examples|Examples]] provide small snippets of functionality to demonstrate how to do useful things with Gaffer's [[:Category:Nodes|Nodes]].

== Templates ==

Templates are more complete scripts, useful as a jumping off point for creating your own Gaffer setups.

== Guides ==

Guides go more in depth into specific aspects of Gaffer. From usage through to customising and extending Gaffer.

* [[Installing Gaffer on AWS with EC2 Image Builder]]

== External Resources ==

[https://www.gafferhq.org/documentation Gaffer's Documentation]

[https://github.com/gafferHQ/gaffer Gaffer GitHub]

[https://www.gafferhq.org/resources GafferHQ Resources]

[https://groups.google.com/g/gaffer-dev Gaffer-Dev Mailing List]

[https://discord.gg/sEm8dDw Gaffer Users Discord]

__NOTOC__

ScatterSamplePrimitiveVariables

2024-06-19T15:28:21Z

Ericmehl: Added Featured Nodes

{{Example
|description=An example of sampling the base surface with a `Scatter` node. In this example, the normals of the surface are sampled onto the scattered points and are used to orient the scattered GafferBots. An additional Z direction is also calculated to make the instanced bots face a focal point.
|file=ScatterSampleSurface.gfr
|screenshots=ScatterSampleSurface.jpg
|categories=Scatter
|nodes=Scatter, Instancer}}

ScatterSamplePrimitiveVariables

2024-06-19T15:19:00Z

Ericmehl: Demonstrates sampling primitive variables in a Scatter node.

File:ScatterSampleSurface.jpg

2024-06-19T15:17:56Z

Ericmehl:

File:ScatterSampleSurface.gfr

2024-06-19T15:13:02Z

Ericmehl: