Introduction
Overview
The Community Shell application (commsh) represents a means of
securing shared, community accounts at the resource level. This document
serves as a guide for resource administrators seeking to implement the
Community Shell on their resources.
The first section, 'Installation', details
basic installation and configuration of the Community Shell.
The second section, 'Administation',
details usage of administrative accounts to manage each community.
Background
A growing trend in supercomputing is the use of community gateways that
provide supercomputing resources to a wide audience. These gateways
typically maintain a user database so that each user can log in to an
individual account at the gateway. However, the community of users may
share a single community account credential which is utilized when
performing computations on supercomputing resources. Thus, while the
gateway/portal may be able to track individuals, the supercomputing
resources can track only the single community account user. Since all
gateway users (theoretically) have access to the community account
credential, a gateway user could conceivably execute unauthorized code on a
supercomputing resource and do so with a high degree of anonymity, which may
result in all users of that gateway losing access to the resource. The
goal of the Community Shell project is to mitigate this potential for abuse
by placing restrictions on the applications which may be executed by a
community account.
To this end, a configuration file is created for each shared account that
explicitly lists which applications (commands) the account can run. It can
further specity the form of the parameters (arguments) passed to those
applications.
In a typical scenario, this configuration file will establish a directory for
applications, known as a Community Software Area, used by the community. Any
applications within this single directory will be allowed, and applications
outside of this directory will be restricted. Applications can then be added
to or removed from this directory as needed.
In a more advanced/secure
scenario, the executables in the Community Software Area are not called
directly by the gateway. Rather, a set of static (i.e. unchanging) scripts
is written by a community developer. These scripts are placed in a
protected location on the supercomputing resources with the restriction that
they are run with security provided by the Community Shell (i.e.
commsh restricts the community account to run only the approved
scripts rather than the binaries in the Community Software Area). The
scripts then call the binaries in the Community Software Area. This allows
for the binaries to be updated as needed (i.e. to upgrade to new software
package versions) without involvement of the system administrator, while
keeping the security restrictions placed on the scripts (so that only a
small set of pre-approved binaries can be executed). The system
administrator is responsible for approving the scripts and installing them
in an appropriate directory.
In either scenario, it is important that the community account itself does
not have access to modify the contents of the Community Software Area. If
the community account credential can be utilized to upload scripts into the
Community Software Area, arbitrary code can be executed on the system
thereby circumventing security precautions. In order to prevent this form of
abuse, it is recommended that a second, community administrator account, be
established for each community.
The community administrator account would be used by the community
administrators to manage which applications the community account can
exeucte. The community administrator account should have its own set of
credentials (password or private key) that are separate from the community
account's. The community administrator account credentials should not be stored on the same server as the
community account credentials, so that a compromise of the community gateway
server will not compromise the administrative account.
It should be noted that this community administrator method is only our
suggested method of managing community accounts. There are other feasible
methods, including group-based access or having the resource administrator
manage each account directly. The method which makes the most sense for a
given resource should be determined by the resource administrator.
Installation
This section explains how to install and configure the Community
Shell (commsh) to restrict community accounts under a UNIX-like system.
Prerequisites
In order to build and install commsh, you will need the following
software installed:
- GNU Make
- A standards-compliant C++ compiler (gcc3 preferred)
- Sed
In order to make use of the GRAM functionality, you will need:
- a fully configured GRAM installation or a GT4.x installation
- GNU Patch (or compatible)
Installation of commsh
A source tarball of the most recent version of commsh can be downloaded
from:
http://security.ncsa.uiuc.edu/research/commaccts/downloads/commsh-latest.tar.gz
You will need at least version 0.1.3b for GRAM support (see below).
After downloading and unpacking the tarball, the package can be built
as follows:
./configure
gmake
gmake install # Note: you must be 'root' to do this
You can control where the package is installed by passing arguments to
./configure. For a full list of options, run:
./configure --help
It is especially important to configure the final location of the
configuration file, as this location is hard-coded into the executable
binary and cannot be changed without recompiling. By default, the
commsh
binary will be installed in /usr/local/bin/ and the associated
configuration file will be read from /etc/commsh.conf. You can
change these locations by using the
"--prefix=/alternate/binary/path"
and "--sysconfdir=/alternate/config/path" command line parameters
when running ./configure.
Proposed Configuration
For a full list of configuration directives, see the commsh.conf man page.
Directories / Files
Here is the proposed layout for the various directories and files utilized
by commsh, a Community Software Area, and a Community Account
User. We assume that the request for a Community Software Area was made by
"jzsmith", who is the primary Community Developer. The
Community Software Area is named "ntroport". The Community Account
User is named "ntrouser". Note that while the Community Software
Area name can differ from the Community Account User name (as shown here),
it is often the case that the two are the same. They are different in this
example to illustrate the ownership of files and directoreis (i.e.
uid and gid).
Ownership Perm Directory / File Usage
--------- ---- ---------------- -----
root:root 0755 /usr/local/bin/commsh Location of commsh binary [1]
root:root 0644 /etc/commsh.conf Configuration file for commsh [2]
root:root 0755 /etc/commsh.d/ Directory for per-community configurations [3]
root:root 0755 /etc/commsh.d/ntroport/ Location of config file and optional scripts for Community Software Area [4]
ntrouser:ntroport 2770 ~ntrouser Community Account User's home directory [5]
jzsmith:ntroport 2775 $TG_COMMUNITY/ntroport/ Location of Community Software Area binaries and associated files [6]
Explanatory Notes
-
The default configuration of
commsh installs the binary into
/usr/local/bin.
While this is what we will use in the proposed configuration, you can change
this with the "--prefix=..." option when running
configure.
-
When
commsh is first installed, a sample configuration file is
installed in
/etc/commsh.conf.sample. The system administrator can look at
this file for some example uses
of commsh. Ultimately, the system administrator must create
/etc/commsh.conf either by copying
the sample configuration file and editing it, or by creating
it from scratch. If you want commsh to read the configuration
file from some place other than /etc/commsh.conf, you can set the
"--sysconfdir=..." option when running configure.
-
This directory is the storage location for all Community Software Area
configuration directories
(as shown in [4] below). In other words, this directory
should contain only directories, no files.
-
The configuration files for
commsh specific to the
ntroport Community Software Area / Community Account User reside
here. The /etc/commsh.conf file contains references to the
various Community Software Areas' configuration files. For example, the
/etc/commsh.conf file will have
entries like this, one for each Community Software Area:
# /etc/commsh.conf
CheckUser ntrouser
ReadUserConfig ntrouser /etc/commsh.d/ntroport/commsh.conf
This new configuration file /etc/commsh.d/ntroport/commsh.conf
has the
entries for binaries (for a simple setup) in the Community Software Area
directory or scripts
(for a complex setup) in the /etc/commsh.d/ntroport directory which will be
called by commsh. Here are two basic configuration files
showing a simple configuration and an advanced configuration.
The first configuration file shows a very simple setup where a Community
Account User is allowed to run any executable in the Community Softare
Account $TG_COMMUNITY/bin
directory, using any number of command line parameters.
# /etc/commsh.d/ntroport/commsh.conf
# Simple configuration - direct access to Community Software Area binaries
# Allow the Community Account User to run any command in the Community Software Area bin directory with any parameters
DirectAccess $TG_COMMUNITY/ntroport/bin/* **
The second example is more complex and involves a bit of indirection to
deter unauthorized modification of allowable executables. In this setup, a
Community Account Developer has created two scripts with the intention that they not be modified.
They are placed in the read-only /etc/commsh.d/ntroport
directory by the system administrator.
Within these two scripts are calls to executables stored in the Community
Software Area
$TG_COMMUNITY/bin directory. (Note that the scripts are highly specific
to a
particular Community Software Area and are not shown here.) The
configuration file below allows
a Community Account User to call only these two scripts with two command
line parameters (for
input and output). The scripts call executables in the Community Software
Area $TG_COMMUNITY/bin
directory with the two command line parameters. The idea here is to give a
Community Account User a limited number of commands that can be called, but
still allow a Community Developer to
update the underlying executables stored in the Community Software Area
$TG_COMMUNITY/bin directory.
# /etc/commsh.d/ntroport/commsh.conf
# Advanced configuration - indirect access to Community Software Area binaries via protected scripts
# Allow the Community Acccount User to run two scripts in the protected directory with two parameters
# script1 and script2 in turn call executables in $TG_COMMUNITY/ntroport/bin
DirectAccess /etc/commsh.d/ntroport/script1 -input * -output *
DirectAccess /etc/commsh.d/ntroport/script2 -input * -output *
-
When the request for a Community Account User has been approved, its
home directory is automatically created. In this case, we deliberately chose
the Community Software Area name to
be different from the Community Account User name. You will probably want to
choose them to be
the same. The Community Account User home directory will be utilized by the various binary files
for input and output since the binaries will be executed with the Community
Account User's
credential via GRAM. In other words, all input files should be transferred
to the Community Account User's home directory, for example via GridFTP. The
Community Software Area binaries, which
are often stored in
$TG_COMMUNITY/ntroport, read the input
files from the Community Acccount User's
home directory. Any output generated by the binaries will be written to the
Community Acccount User's home directory. This output can then be fetched using GridFTP. Note
that the group permission (gid) on the Community Acccount User's home directory allows for easy
access by the Community Developers for debugging purposes.
-
When the request for a Community Software Area has been approved, the
Community Developers's Community Software Area directory is
automatically created. Note again that we deliberately chose the Community
Software Area name to
be different from the Community Account User name, but you should choose them to be the same.
The directory
$TG_COMMUNITY/ntroport is where the binaries (and other
associated files) for the Community Software Area will be stored. These binaries are called
either directly or by the scripts located in
/etc/commsh.d/ntroport/. This
allows for the binaries to be updated frequently while keeping the scripts
secure (since the scripts should not require frequent updating).
Here we show a basic configuration for two Community Account Users named
commuser1 and commuser2. Each Community Account
User will have an entry in the global /etc/commsh.conf file to
read in the per-community configuration files.
/etc/commsh.conf:
# Allow any user but root to run commands through commsh
AllowUser *
DenyUser root
# Community Acccount User 1 (commuser1)
# Check GRAM job submissions by this user, assuming GRAM support is enabled
CheckUser commuser1
# Load an external configuration file for this user
ReadUserConfig commuser1 /etc/commsh.d/commuser1/commuser1.conf
# Community Acccount User 2 (commuser2)
# Check GRAM job submissions by this user, assuming GRAM support is enabled
CheckUser commuser2
# Load an external configuration file for this user
ReadUserConfig commuser2 /etc/commsh.d/commuser2/commuser2.conf
/etc/commsh.d/commuser1/commuser1.conf:
# Allow commuser1 to run any command in /opt/communities/commuser1/bin with any parameters
DirectAccess /opt/communities/commuser1/bin/* **
# Allow commuser1 to create new directories and delete any files in /home/commuser1/shared/
DirectAccess /bin/mkdir /home/commuser1/shared/*
DirectAccess /bin/rm -rf /home/commuser1/shared/*
/etc/commsh.d/commuser2/commuser2.conf:
Allow commuser2 to run three scripts with partially specified parameters.
DirectAccess /opt/communities/commuser2/bin/script1 -input * -output *
DirectAccess /opt/communities/commuser2/bin/script2 -input * -output *
DirectAccess /opt/communities/commuser2/bin/script3 -input * -output *
The most important thing to understand when creating command
restrictions is the difference between * and
**, and what each of them will match.
A single asterisk (*) will match any character in a
single argument.
In general, this means it will not match a space unless the space is
enclosed in quotation marks or escaped with a back-slash. Additionally,
an asterisk in the command itself will not match a backslash
(/). For example:
DirectAccess /opt/communities/*/script < *
Will match:
/opt/communities/one/script < input/opt/communities/two/script < "input file"/opt/communities/three/script < input\ file
But will not match:
/opt/communities/one/two/script < input/opt/communities/one/script < input file
In contrast, a double asterisk (**) should only appear
at the end of a command restriction specification, and indicates that
any additional parameters will be
accepted.
Activation / Shell Support
In order to activate commsh parsing for a given user, the user's shell
must be changed to commsh. Exact details on how to change a user's shell
vary from system to system, but is often set in /etc/passwd. The Community Account User's shell should be changed
to:
/usr/local/bin/commsh
Or to the location where you installed commsh, if you specified a
different installation location.
Typically, the shell must be added to /etc/shells (or a
similar file) for the system to accept it as a valid shell.
GRAM support (optional)
GRAM support of commsh is currently part of the latest
standard GRAM
installation. However, older versions of GRAM require a simple patch to make it possible to use
commsh
to implement command-based restrictions on GRAM jobs. This patch can
be downloaded from:
http://security.ncsa.uiuc.edu/research/commaccts/downloads/globus-job-manager-script-pl.diff
To install the patch, run the following as user globus:
cp globus-job-manager-script-pl.diff $GLOBUS_LOCATION/libexec
cd $GLOBUS_LOCATION/libexec
patch -p0 < globus-job-manager-script-pl.diff
If the patch complains about 'fuzz', don't worry. The patch is designed
to work with multiple versions of GRAM. As long as both hunks succeed,
the patch has been successfully applied.
Once the patch is applied, edit
$GLOBUS_LOCATION/libexec/globus-job-manager-script.pl and
uncomment the $FILTER_COMMAND line. Also change this line to
point to your
installation of commsh if you changed the installation location.
NOTE: Under some installations, such as TG CTSS, there are separate
trees for each service. Under these installations, the patch must be
applied to all directories serving GRAM, both WSGRAM (GRAM4) and
pre-WSGRAM (GRAM2) services. Under TG CTSS, this should be the
prews-gram directory as well as the globus-wsrf
directory.
Other Links
Man page for commsh binary
Man page for commsh.conf configuration file
Community
Shell Installation / Configuration Instructions for TeraGrid
