Overview of sam_common_pylib

This documentation was generated by and for sam_common_pylib v8_6_0.
See the release notes.

The package structure of the common python classes/tools has been revamped for the release of sam_common_pylib. This document describes (briefly) the main components of each of the packages within sam_common_pylib.

General Description:

sam_common_pylib is meant to include all of the common tools, classes, scripts, etc., used for interfacing to SAM via python. Higher level applications (such as sam_user_pyapi, sam_admin_pyapi and sam_mis_pyapi) make extensive use of the tool suite available in sam_common_pylib.

Somewhat out-of-date (but still pretty accurate) documentation on how to build and run a sam_db_srv using the sam_common_pylib tools is available.

Package History:

As of Spring 2005, the sam_common_pylib package is being deployed as part of the SAM v6 upgrade. It replaces the old sam_common package, which will become obsolete as soon as there are no more v5 clients.

Package czar: Lauri Loebel Carpenter. (lauri@fnal.gov)

Package Organization:

sam_common_pylib is organized into the following packages:

SamCommand
Contains the tools and utilities for parsing and dispatching commands; see CommandInterface.html for details.

SamUtility
Catch-all holding place for useful utilities, functions, scripts, etc., that don't fit anywhere else. At the time of this writing, these may be summarized by:

Definitely Supported:

  • GnuPlot.py: plotting tools used to generate various sam plots; see GnuPlot.html for more details.
  • DbDerivedClient.py: Implementation of the client-side "get", "getOne", and "query", as well as "getAttrList" (using the dbDerived classes in the sam_db_srv package). The "query" interface has been implemented in sam_user_pyapi as the "sam db query" command (as an example of use), but "get" and "getOne" do not lend themselves to interactive (command-line) use.
  • CaseInsensitiveDictionary.py: implementation of a case-insensitive python dictionary, used for metadata parameters and command-line parsing.
  • ConnectStringObj.py: class to handle the oracle username/password strings entered by shifters for protected commands.
  • MetadataRequirementHandler.py: utility for managing experiment-specific (e.g., db-server-specific) metadata attribute requirements (such as: which attributes are required for which kinds of files, etc.).
  • MethodWrapper.py: Utility to "intercept" calls to a class object's methods; used by the sam_db_srv/ServantFactory to provide automarshalling servants.
  • CvsPackageHandler.py: class to handle the CVS issues related to the maintenance of the sam .k5login files.
  • DictFormatter.py: utility for pretty printing of python dictionaries (heavily used in the repr and str methods of the python class wrappers)
  • MailMsgObj.py: utility for sending mail
  • SamUnitTest.py: Base classes that can be used in writing test suites, etc.
  • SamUserIdentifier.py: utility to provide centralized method to get your current user identifier
  • SamHostName.py: utility to provide centralized method to get the current host name.
  • FastCgiHandler.py: wrapper around fcgi for faster web script initialization
  • fcgi.py: interacts with apache webserver for fast cgi scripts.
  • SamHTMLgen.py: SAM-ified HTMLgen, for writing SAM docs. Used by samAtAGlance and other cron job report writers.

Unknown Support Status: (i.e., copied into this directory from the old package sam_common, but not necessarily used or supported or known to work....)

  • client_util.py: many obsolete (but probably still used) functions from the old sam_user days
  • CommonUtility.py: I do not know what this is or who uses it.
  • ModuleLoader.py: don't know what this is, or who is using it.
  • RequestRequirementHandler.py: Carmenita was working on this, I think it was modeled after the MetadataRequirementHandler, but requests may not need this level of details....
  • SamEnvironment.py: don't know what this is, or who is using it.
  • SAMLog.py: don't know what this is or who is using it.
  • SamTimer.py: legacy from ancient versions of sam_common, this could probably be removed.
  • Statistics.py: don't know what this is or who uses it
  • subprocess.py: don't know what this is or who uses it

SamCorba
Contains all code related specifically to the CORBA interfaces of SAM: ORB-specific code, the code for the server proxies that are used to talk to other SAM entities, and the code for marshalling/unmarshalling CORBA datastreams. Note: the DbCorbaClient.py module (which used to be in this directory merely because it has the string 'Corba' in the name) has been moved and renamed to SamUtility/DbDerivedClient.py).

Specific files of interest are:

  • SAMCorbaClient.py:
    • all ORB-specific code, and the DBserver servant factory, is here. Any application which needs to reference CORBA objects should import CORBA from this file.
  • Marshalling (corbaRepr) and Unmarshalling (pyRepr):
    • SAMToSamStructCodeGenerator.py: script called at build-time (by the GNUmakefile) to generate the mapping from CORBA structures/unions to python class wrappers. This script imports the SAM, SAMStation, and SAMDbServer modules (from sam_idl_pylib), inspects the namespaces of each module, and looks for python modules in sam_common_pylib that "match" our naming conventions and are presumed to be the class wrapper for that CORBA object. NOTE that there are a lot of assumptions about our naming conventions here!
    • SAMToSamStruct.py: the mapping from CORBA objects to python class wrappers, used when unmarshalling CORBA data streams.
    • SamIdlStructWrapperBase.py:
      The functions "marshal" and "unmarshal" are defined here, as well as the base classes of all derived python wrapper classes.
  • Server proxy code:
    • SamServerProxyManager.py: Class which can be used to manage a collection of proxies, so that any particular proxy is only instantiated once (and then reused as necessary).
    • SamServerProxyRetryHandler.py: class which handles automatic retries for all calls to any interface.
    • SamServerProxy.py: base classes defining the characteristics of all SAM server proxies, including automarshalling capabilities and retry handlers.
    • BasePythonServerProxy.py: base class for all clients talking to sam_server_pylib-derived servers (sam_db_srv, sam_dimension_server_prototype, sam_pnfs_srv).
    • StationProxy.py: derived classes which represent the specific proxies to all interfaces defined in sam_idl/sam_station/*.idl
    • OptimizerProxy.py: derived class which represents the interface to the optimizer.
    • DbServerProxy.py: derived classes which represent the specific proxies to all interfaces defined in sam_idl/sam_db_srv/*.idl
    • DimensionServerProxy.py: derived classes which represent the specific proxies to all interfaces defined in the sam_idl/sam_dimension_server/*.idl
    • PnfsServerProxy.py: derived classes which represent the specific proxies to all interfaces defined in the sam_idl/sam_pnfs_srv/*.idl

SamException

Contains the code for dealing with exceptions in SAM code. Specific files of interest are:
  • SamBaseException.py: Base class for all derived SAM exceptions.
  • SamExceptionCodeGenerator.py: script called at build-time (by the GNUmakefile) to generate the mapping from declared CORBA exceptions (in sam_idl/sam_corba_base/SamExceptions.idl) to corresponding python exception classes (sam_common_pylib/SamException/SamExceptions.py).
  • SamExceptionHandler: class intended to handle exceptions from within client code.

SamStruct
Contains the python class wrappers for (nearly) all objects defined in sam_idl/sam_corba_base. Class wrappers are at the heart of the sam infrastructure; they are python classes which can be initialized in several ways (from an existing instance, from a CORBA data stream, from 'sensible' input arguments, etc.), and which have defined the "pyRepr()" and "corbaRepr()" methods used by the automarshalling sam server proxies (and by the auto-marshalling sam db server servants).

Specific items of interest are:

  • TEMPLATE_STRUCT_WRAPPER
    • template file, with lots of comments, for constructing a new python class wrapper.
  • SamStructSelftestBaseClass.py:
    • Base class for the capability to run the selftest for each wrapper class.

SamStationStruct
Contains the python class wrappers for various sam station structures (from sam_idl/sam_station or other locations).

SamDimensionStruct
Contains the python class wrappers for the data structures used in the sam_dimension_server (from sam_idl/sam_dimension_server).

SamLegacyStruct
Contains the python class wrappers for legacy structures (from sam_idl/sam_station or other locations) as needed. Hopefully not too many of them.

SamFile
Contains the python class wrappers and utilities for Sam Metadata structures. Metadata is complicated by the need for experiment-specific metadata requirements; hence, in order to determine which attributes of metadata are required for each file type, we need to contact the dbserver (wherein is maintained the experiment-specific data). The

SamConfig
Don't really know what this stuff is, but it was related to sam_config evidently, hence was moved into this directory. Don't know if it's being used or not.

Author: Lauri Loebel Carpenter
Last update by: $Author: lauri $
Updated on: $Date: 2005/06/09 16:37:27 $