thredds.inventory
Class DatasetCollectionManager

java.lang.Object
  thredds.inventory.DatasetCollectionManager

All Implemented Interfaces:

CollectionManager

Direct Known Subclasses:

DatasetCollectionFromCatalog

@ThreadSafe
public class DatasetCollectionManager
extends Object
implements CollectionManager

Manage one or more Directory Scanners that find MFiles Keep track of when they need to be rescanned. Used in:

• replaces older version in ncml.Aggregation
• ucar.nc2.ft.point.collection.TimedCollectionImpl

looks like we need to be thread safe, for InvDatasetFeatureCollection

Since:

Jul 8, 2009

Author:

caron

Nested Class Summary

class	DatasetCollectionManager.TriggerEvent
static interface	DatasetCollectionManager.TriggerListener

Field Summary

static String	CATALOG
protected String	collectionName
protected DateExtractor	dateExtractor
static String	PROTO
static String	RESCAN

Constructor Summary

protected	DatasetCollectionManager()
	DatasetCollectionManager(FeatureCollectionConfig config, Formatter errlog)
	DatasetCollectionManager(String recheckS) For retrofitting to Aggregation Must also call addDirectoryScan one or more times

Method Summary

void	addDirectoryScan(String dirName, String suffix, String regexpPatternString, String subdirsS, String olderS, Object auxInfo) Add a directory scan to the collection
void	addEventListener(DatasetCollectionManager.TriggerListener l)
void	close() release any resources
static void	enableMetadataManager()
Date	extractRunDate(MFile mfile) Use the date extractor to extract the date from the filename
String	getCollectionName()
CollectionSpecParser	getCollectionSpecParser()
List<MFile>	getFiles() Get the current collection of MFile, since last scan or rescan.
long	getLastChanged() Get the last time files changed
long	getLastScanned() Get the last time scanned
byte[]	getMetadata(MFile file, String key)
double	getOlderThanFilterInSecs()
int	getProtoIndex()
TimeUnit	getRecheck() Get how often to rescan
String	getRoot() Get common root of all MFiles in the collection - may be null
boolean	hasDateExtractor()
protected boolean	hasScans()
boolean	isRescanNeeded() Compute if rescan is needed, based on the recheckEvery parameter.
static DatasetCollectionManager	open(String collection, String olderThan, Formatter errlog)
void	putMetadata(MFile file, String key, byte[] value)
void	removeEventListener(DatasetCollectionManager.TriggerListener l)
boolean	rescan() Rescan the collection.
boolean	rescanIfNeeded()
void	resetProto()
void	scan(CancelTask cancelTask) Scan the directory(ies) and create MFile objects.
protected void	scan(Map<String,MFile> map, CancelTask cancelTask)
static void	setController(MController _controller) Set the MController used by scan.
void	setDateExtractor(DateExtractor dateExtractor)
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

CATALOG

public static final String CATALOG

See Also:

Constant Field Values

RESCAN

public static final String RESCAN

See Also:

Constant Field Values

PROTO

public static final String PROTO

See Also:

Constant Field Values

collectionName

protected String collectionName

dateExtractor

protected DateExtractor dateExtractor

Constructor Detail

DatasetCollectionManager

public DatasetCollectionManager(FeatureCollectionConfig config,
                                Formatter errlog)

DatasetCollectionManager

protected DatasetCollectionManager()

DatasetCollectionManager

public DatasetCollectionManager(String recheckS)

For retrofitting to Aggregation Must also call addDirectoryScan one or more times

Parameters:

recheckS - a undunit time unit, specifying how often to rscan

Method Detail

enableMetadataManager

public static void enableMetadataManager()

setController

public static void setController(MController _controller)

Set the MController used by scan. Defaults to thredds.filesystem.ControllerOS() if not set.

Parameters:

_controller - use this MController

open

public static DatasetCollectionManager open(String collection,
                                            String olderThan,
                                            Formatter errlog)
                                     throws IOException

Throws:

IOException

close

public void close()

Description copied from interface: CollectionManager

release any resources

Specified by:

close in interface CollectionManager

setDateExtractor

public void setDateExtractor(DateExtractor dateExtractor)

addDirectoryScan

public void addDirectoryScan(String dirName,
                             String suffix,
                             String regexpPatternString,
                             String subdirsS,
                             String olderS,
                             Object auxInfo)

Add a directory scan to the collection

Parameters:

dirName - scan this directory

suffix - require this suffix (overriddden by regexp), may be null

regexpPatternString - if present, use this reqular expression to filter files , may be null

subdirsS - if "true", descend into subdirectories, may be null

olderS - udunit time unit - files must be older than this amount of time (now - lastModified > olderTime), may be null // * @param dateFormatString dateFormatMark string, may be null

auxInfo - attach this object to any MFile found by this scan

getCollectionName

public String getCollectionName()

Specified by:

getCollectionName in interface CollectionManager

getRoot

public String getRoot()

Description copied from interface: CollectionManager

Get common root of all MFiles in the collection - may be null

Specified by:

getRoot in interface CollectionManager

Returns:

msecs since 1970

getCollectionSpecParser

public CollectionSpecParser getCollectionSpecParser()

getOlderThanFilterInSecs

public double getOlderThanFilterInSecs()

getRecheck

public TimeUnit getRecheck()

Get how often to rescan

Specified by:

getRecheck in interface CollectionManager

Returns:

time dureation of rescan period, or null if none.

getLastScanned

public long getLastScanned()

Get the last time scanned

Specified by:

getLastScanned in interface CollectionManager

Returns:

msecs since 1970

getLastChanged

public long getLastChanged()

Get the last time files changed

Specified by:

getLastChanged in interface CollectionManager

Returns:

msecs since 1970

scan

public void scan(CancelTask cancelTask)
          throws IOException

Scan the directory(ies) and create MFile objects. Get the results from getFiles()

Specified by:

scan in interface CollectionManager

Parameters:

cancelTask - allow user to cancel

Throws:

IOException - if io error

rescanIfNeeded

public boolean rescanIfNeeded()
                       throws IOException

Throws:

IOException

hasScans

protected boolean hasScans()

isRescanNeeded

public boolean isRescanNeeded()

Compute if rescan is needed, based on the recheckEvery parameter. True if scanList not empty, recheckEvery not null, and recheckEvery time has passed since last scan

Specified by:

isRescanNeeded in interface CollectionManager

Returns:

true if rescan is needed

rescan

public boolean rescan()
               throws IOException

Rescan the collection. Files may have been deleted or added. If the MFile already exists in the current list, leave it in the list. If returns true, get the results from getFiles(), otherwise nothing has changed.

Specified by:

rescan in interface CollectionManager

Returns:

true if anything actually changed.

Throws:

IOException - on I/O error

getFiles

public List<MFile> getFiles()

Get the current collection of MFile, since last scan or rescan. Must call scan() or rescan() at least once

Specified by:

getFiles in interface CollectionManager

Returns:

current list of MFile, sorted by name

extractRunDate

public Date extractRunDate(MFile mfile)

Description copied from interface: CollectionManager

Use the date extractor to extract the date from the filename

Specified by:

extractRunDate in interface CollectionManager

Parameters:

mfile - extract from here

Returns:

Date, or null if none

hasDateExtractor

public boolean hasDateExtractor()

scan

protected void scan(Map<String,MFile> map,
                    CancelTask cancelTask)
             throws IOException

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

getProtoIndex

public int getProtoIndex()

resetProto

public void resetProto()

Specified by:

resetProto in interface CollectionManager

addEventListener

public void addEventListener(DatasetCollectionManager.TriggerListener l)

removeEventListener

public void removeEventListener(DatasetCollectionManager.TriggerListener l)

putMetadata

public void putMetadata(MFile file,
                        String key,
                        byte[] value)

Specified by:

putMetadata in interface CollectionManager

getMetadata

public byte[] getMetadata(MFile file,
                          String key)

Specified by:

getMetadata in interface CollectionManager