lsst::ap::detail::SubManager< MutexT, DataT, TraitsT > Class Template Reference

Helper class for managing chunks of a particular type. More...

#include <ChunkManagerImpl.h>

Inheritance diagram for lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >:

Public Types
typedef BlockAllocator< MutexT, DataT, TraitsT >	Allocator
typedef ChunkRef< Allocator, DataT, TraitsT >	Chunk
typedef ChunkDescriptor < TraitsT::MAX_BLOCKS_PER_CHUNK >	Descriptor

Public Member Functions
	SubManager (unsigned char const *const ref, std::size_t const offset)
int	size () const
	Returns the number of chunks under management. More...
int	space () const
	Returns the number of additional chunks that could be handled by this manager. More...
void	createOrRegisterInterest (std::vector< Chunk > &toRead, std::vector< Chunk > &toWaitFor, int const visitId, std::vector< int > const &chunkIds)
bool	checkForOwnership (std::vector< Chunk > &toRead, std::vector< Chunk > &toWaitFor, int const visitId)
void	getChunks (std::vector< Chunk > &chunks, std::vector< int > const &chunkIds)
bool	relinquishOwnership (int const visitId, bool const rollback, VisitTracker const &tracker)
void	print (std::ostream &os) const
void	print (int const chunkId, std::ostream &os) const
void	printVisit (int const visitId, std::ostream &os) const

Public Attributes
HashedSet< Descriptor, NUM_CHUNKS >	_chunks
Allocator	_allocator

Static Public Attributes
static int const	NUM_CHUNKS = TraitsT::MAX_CHUNKS_PER_FOV * MAX_VISITS_IN_FLIGHT

Detailed Description

template<typename MutexT, typename DataT, typename TraitsT = DataTraits<DataT>>
class lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >

Helper class for managing chunks of a particular type.

To be used exclusively by chunk manager implementations.

Definition at line 219 of file ChunkManagerImpl.h.

Member Typedef Documentation

template<typename MutexT , typename DataT , typename TraitsT = DataTraits<DataT>>

typedef BlockAllocator<MutexT, DataT, TraitsT> lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::Allocator

Definition at line 221 of file ChunkManagerImpl.h.

template<typename MutexT , typename DataT , typename TraitsT = DataTraits<DataT>>

typedef ChunkRef<Allocator, DataT, TraitsT> lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::Chunk

Definition at line 222 of file ChunkManagerImpl.h.

template<typename MutexT , typename DataT , typename TraitsT = DataTraits<DataT>>

typedef ChunkDescriptor<TraitsT::MAX_BLOCKS_PER_CHUNK> lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::Descriptor

Definition at line 223 of file ChunkManagerImpl.h.

Constructor & Destructor Documentation

template<typename MutexT , typename DataT , typename TraitsT = DataTraits<DataT>>

lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::SubManager ( unsigned char const *const  ref, std::size_t const  offset  )

inline

Definition at line 234 of file ChunkManagerImpl.h.

: _allocator(ref, offset) {}

Member Function Documentation

template<typename MutexT , typename DataT , typename TraitsT >

bool lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::checkForOwnership	(	std::vector< Chunk > &	toRead,
		std::vector< Chunk > &	toWaitFor,
		int const	visitId
	)

Checks to see whether the chunks in the toWaitFor list are owned by the given visit. Any chunks that have had their ownership transferred to the given visit are removed from toWaitFor. Of these chunks, the subset that were not completely read into memory (that is, whose previous owners failed while reading them in) are appended to toRead.

Parameters

[out]	toRead	Set to the list of chunks now owned by the given visit, but which must be re-read from disk.
[in,out]	toWaitFor	The list of chunks for which ownership must be checked. Any chunks now owned by the given visit are removed from the list by the call.
[in]	visitId	An identifier for a visit to a FOV.

Returns

true if and only if all the chunks initially in the toWaitFor list now belong to the given visit.

Definition at line 464 of file ChunkManagerImpl.cc.

  {
    std::size_t size = toWaitFor.size();
    std::size_t i = 0;

    while (i < size) {
        Chunk & c = toWaitFor[i];
        if (c.getVisitId() != visitId) {
            ++i;
        } else {
            if (!c.isUsable()) {
                c.clear();
                toRead.push_back(c);
            }
            // deletes element i in O(1) time (but changes element ordering)
            --size;
            if (i < size) {
                toWaitFor[i] = toWaitFor[size];
            }
            toWaitFor.pop_back();
        }
    }
    return toWaitFor.empty();
}

template<typename MutexT , typename DataT , typename TraitsT >

void lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::createOrRegisterInterest	(	std::vector< Chunk > &	toRead,
		std::vector< Chunk > &	toWaitFor,
		int const	visitId,
		std::vector< int > const &	chunkIds
	)

Registers the given visit as an interested party of each of the given chunks. If any of the given identifiers doesn't correspond to a chunk, an empty chunk is created. Newly created chunks are stored in the toRead list (indicating data for them must be read from disk), previously existing chunks are returned in the toWaitFor list (indicating that the visit must wait until it owns those instances before processing can begin).

Parameters

[out]	toRead	Set to the list of chunks that were not found in memory and must be read in from disk.
[out]	toWaitFor	Set to the list of chunks instances that are already in memory and must be waited on.
[in]	visitId	The visit to register interest or create chunks for.
[in]	chunkIds	A list of identifiers for chunks required by the visit. Assumed to be duplicate free.

Exceptions

lsst::pex::exceptions:::MemoryError

Thrown if there is insufficient space to track all requested chunks.

Definition at line 420 of file ChunkManagerImpl.cc.

  {
    std::vector<int>::const_iterator const end = chunkIds.end();
    for (std::vector<int>::const_iterator i = chunkIds.begin(); i != end; ++i) {
        std::pair<Descriptor *, bool> p(_chunks.findOrInsert(*i));
        if (p.second) {
            // new chunk descriptor was allocated
            if (p.first == 0) {
                throw LSST_EXCEPT(lsst::pex::exceptions::MemoryError,
                                  "too many chunks in flight");
            }
            p.first->_visitId = visitId;
            p.first->_usable  = false;
            toRead.push_back(Chunk(p.first, &_allocator));
        } else {
            // existing chunk descriptor was found
            assert(p.first != 0);
            p.first->_interestedParties.enqueue(visitId);
            toWaitFor.push_back(Chunk(p.first, &_allocator));
        }
    }
}

template<typename MutexT , typename DataT , typename TraitsT >

void lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::getChunks	(	std::vector< Chunk > &	chunks,
		std::vector< int > const &	chunkIds
	)

Returns a chunk for each of the given identifiers that corresponds to a chunk managed by this SubManager.

Parameters

[out]	chunks	Set to the list of chunk instances managed by this SubManager and with an identifier in chunkIds.
[in]	chunkIds	The list of chunk identifiers to return chunk instances for. Assumed to be duplicate free.

Definition at line 503 of file ChunkManagerImpl.cc.

  {
    std::vector<int>::const_iterator const end = chunkIds.end();
    for (std::vector<int>::const_iterator i = chunkIds.begin(); i != end; ++i) {
        Descriptor * d = _chunks.find(*i);
        if (d != 0) {
            chunks.push_back(Chunk(d, &_allocator));
        }
    }
}

template<typename MutexT , typename DataT , typename TraitsT >

void lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::print

(

std::ostream &

os

)

const

Definition at line 627 of file ChunkManagerImpl.cc.

                                                                    {
    Descriptor const * const end = _chunks.end();
    std::vector<Descriptor const *> v;
    v.reserve(_chunks.size());
    for (Descriptor const * beg = _chunks.begin(); beg != end; ++beg) {
        if (beg->_chunkId != -1) {
            v.push_back(beg);
        }
    }
    std::sort(v.begin(), v.end(), PtrLessThan<Descriptor>());
    os << "    Chunks with an owner";
    if (v.empty()) {
        os << ": None";
    } else {
        os << ":\n";
        os << "    Owned by visit " << v[0]->_visitId << ":\n";
        printChunks(os, v);
    }
    os << std::endl;
}

template<typename MutexT , typename DataT , typename TraitsT >

void lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::print	(	int const	chunkId,
		std::ostream &	os
	)		const

Definition at line 650 of file ChunkManagerImpl.cc.

                                                                                       {
    Descriptor const * c = _chunks.find(chunkId);
    os << "    [" << c->_chunkId << "] chunk " <<
          ZoneStripeChunkDecomposition::chunkToSequence(chunkId) << " in stripe " <<
          ZoneStripeChunkDecomposition::chunkToStripe(chunkId);
    if (c == 0) {
        os << ": not being tracked\n";
    } else {
        os << ":\n        " << (c->_usable ? "usable" : "unusable") << "\n        ";
        if (c->_interestedParties.empty()) {
            os << "un";
        }
        os << "interesting\n        ";
        int sz = c->_size;
        os << sz << " entries in " << c->_nextBlock << " blocks (" <<
              c->_numBlocks << " allocated)\n        ";
        if (sz <= c->_delta) {
            sz = 0;
        } else {
            sz -= c->_delta;
        }
        os << sz << " entries in delta\n";
    }
    os << std::endl;
}

template<typename MutexT , typename DataT , typename TraitsT >

void lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::printVisit	(	int const	visitId,
		std::ostream &	os
	)		const

Definition at line 678 of file ChunkManagerImpl.cc.

                                                                                            {
    Descriptor const * const end = _chunks.end();
    std::vector<Descriptor const *> v;
    v.reserve(_chunks.size());
    for (Descriptor const * beg = _chunks.begin(); beg != end; ++beg) {
        if (beg->_chunkId != -1 && beg->_visitId == visitId) {
            v.push_back(beg);
        }
    }
    std::sort(v.begin(), v.end(), PtrLessThan<Descriptor>());
    os << "    Chunks belonging to visit " << visitId;
    if (v.empty()) {
        os << ": None";
    } else {
        os << ":\n";
        printChunks(os, v);
    }
    os << std::endl;
}

template<typename MutexT , typename DataT , typename TraitsT >

bool lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::relinquishOwnership	(	int const	visitId,
		bool const	rollback,
		VisitTracker const &	tracker
	)

Relinquishes ownership of any chunks owned by the given visit (each chunk is passed on to its first interested party that is still in flight).

Parameters

[in]	visitId	The visit owning the chunks to relinquish ownership of.
[in]	rollback	Flag indicating whether or not in-memory changes to a chunk should be rolled back (true) or committed (false) prior to relinquishing ownership.
[in]	tracker	Tracks the status of visits (whether or not a visit is in flight, and if so, whether or not it has failed).

Returns

true if any chunks changed hands.

Definition at line 531 of file ChunkManagerImpl.cc.

  {
    bool change = false;
    Descriptor * const end = _chunks.end();
    for (Descriptor * i = _chunks.begin(); i != end; ++i) {
        if (i->getId() != -1 && i->_visitId == visitId) {
            bool foundSuccessor = false;
            while (!i->_interestedParties.empty()) {
                int const nextVisitId = static_cast<int>(i->_interestedParties.dequeue());
                if (tracker.isValid(nextVisitId)) {
                    i->_visitId    = nextVisitId;
                    change         = true;
                    foundSuccessor = true;
                    break;
                }
            }
            if (foundSuccessor) {
                Chunk c(i, &_allocator);
                if (rollback) {
                    c.rollback();
                } else {
                    c.commit();
                }
            } else {
                // deallocate chunk
                _allocator.free(i->_blocks, i->_numBlocks);
                _chunks.erase(i->getId());
            }
        }
    }
    return change;
}

template<typename MutexT , typename DataT , typename TraitsT = DataTraits<DataT>>

int lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::size ( ) const

inline

Returns the number of chunks under management.

Definition at line 237 of file ChunkManagerImpl.h.

{ return _chunks.size();  }

template<typename MutexT , typename DataT , typename TraitsT = DataTraits<DataT>>

int lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::space ( ) const

inline

Returns the number of additional chunks that could be handled by this manager.

Definition at line 239 of file ChunkManagerImpl.h.

{ return _chunks.space(); }

Member Data Documentation

template<typename MutexT , typename DataT , typename TraitsT = DataTraits<DataT>>

Allocator lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::_allocator

Definition at line 230 of file ChunkManagerImpl.h.

template<typename MutexT , typename DataT , typename TraitsT = DataTraits<DataT>>

HashedSet<Descriptor, NUM_CHUNKS> lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::_chunks

Definition at line 229 of file ChunkManagerImpl.h.

template<typename MutexT , typename DataT , typename TraitsT = DataTraits<DataT>>

int const lsst::ap::detail::SubManager< MutexT, DataT, TraitsT >::NUM_CHUNKS = TraitsT::MAX_CHUNKS_PER_FOV * MAX_VISITS_IN_FLIGHT

static

Definition at line 227 of file ChunkManagerImpl.h.

---

The documentation for this class was generated from the following files:

• /home/lsstsw/stack/Linux64/ap/10.0+224/include/lsst/ap/ChunkManagerImpl.h
• /home/lsstsw/stack/Linux64/ap/10.0+224/include/lsst/ap/ChunkManagerImpl.cc