High Performance Computing

LONI User Allocation and Account Policy

Last revised: 30 July 2024.

1. Accounts and Allocations

LONI (Louisiana Optical Network Infrastructure) maintains a high speed optical network that connects most of the institutions of higher education in the State of Louisiana as well as several other entities. LONI also provides high performance computing (HPC) systems to support research, education, and economic development in the State. Detailed information about LONI can be found on the LONI web site. LONI controls access to these resources via a formal user account and resource allocation process. All decisions in these matters are made by the LONI Resource Allocation Committee (LRAC) made up of representives from each of the LONI member organizations.

Gaining access to LONI resources, be it computer time or data storage, involves a two step process. Since all interactions with the systems are controlled by user account, the first step requires placing a request for one. Only one account is needed to access any of the LONI systems, as well as the process management tools provided on the various web sites. All faculty and research staff are eligible for user accounts, which may be applied for on-line.. Be aware that all usage is governed by the LONI Usage Policy. Student and research collaborators are eligible for accounts, but they must be sponsored by a faculty member.

The second step in the access process is to associate a user account with a resource allocation. Compute time is a finite resource, so access is controlled by an allocation process. An allocation provides a block of computational time, measured in Service Units of 1 wallclock hour on 1 CPU processor core. Likewise, storage space (on some systems) is controlled by a quota system, and allows for storing and sharing large data sets. The remainder of this document will discuss allocations, while use of storage is discussed in more detail in the LONI Storage Policy document.

Allocations are awarded through a proposal submitted by a principle investigator (PI). Full-time faculty and research staff at LONI Member and Associate institutions are eligible to serve as a PI for allocation purposes. Resource allocations for undergraduate and graduate students should be requested by a faculty advisor. The LONI Management Council reserves the right to designate others for PI eligibility. Once awarded, the PI is able to authorize other users to charge resource consumption against the allocation. The PI is responsible for all authorizations, but may formally delegate some other user to manage this process. Once a user account is authorized to use an allocation, the user is enabled to submit jobs or otherwise expend the allocation resources. This implies that a user must be authorized for at least one non-expired allocation before the systems can be used. Likewise, a user account may be authorized to use multiple allocations. It becomes the responsibility of the PI and user to make sure the appropriate allocation is charged for work.

The second step involves associating user accounts with specific allocations against which they can charge resource usage. To accomplish this, tools are made available to the PI to control authorization of users on their allocations. The PI associates an email address with an allocation to which all user account requests are sent for authorization. The PI is responsible for all authorizations, but may formally delegate someone to manage this process. Once a user account is authorized to use an allocation, the user is allowed to submit jobs or otherwise expend the allocation. This implies that a user must be authorized for at least one non-expired allocation before the systems can be used. Likewise, a user account may be authorized to use multiple allocations. It becomes the joint responsibility of the PI and user to make sure the appropriate allocation is charged for work performed.

A single machine will have multiple nodes (individual servers) available, with multiple cores within each node. This allows many cores to be used for a single parallel processing job, but can lead to not-so-obvious charges. For example, running for 1 hour using 8 cores on an 8-core node consumes 8 SU's. On the other hand, running for 1 hour on 1 core of the same 8 core node, which allows the single core to access all of the node memory, also consumes 8 SU's. In simple terms, the number of cores that are reserved for a job, and hence are unavailable to others, is the number used to calculate SU usage.

2. The Allocation Process

LONI resources are partitioned by category, and all allocation proposals go through the LONI Resource Allocation Committee (LRAC). The LRAC represents three distinct approval authorities: individual member institutions; the LONI Management Council (LMC), represented by the LONI director; and the LRAC as a panel. Table 1 shows the distribution of LONI resources by category:

All allocations have a duration of one year, and are considered active until they expire regardless of current balance. They are considered valid so long as they have a positive balance and have not yet expired. Start-up Allocations may be awarded at any time, and Large Allocations are granted at the beginning of every calendar quarter. Application deadlines for Large Allocations are one month prior to the start date, as summarized in Table 2.

A PI may request, or be requested, to make a formal presentation to the allocation authority in conjunction with their proposal submission.

To facilitate management of proposals, they are identified as belonging to one of several classes (Table 3).

Higher charge for GPU nodes: On the QB-4 cluster, jobs running on the GPU nodes are charged for 64 SUs for each GPU-hour they consume. For instance, if a job runs two hours with one GPU device, it will be charged 2 hours * 1 GPU * 64 = 128 SUs. PIs should consider the extent of GPU node use while preparing allocation requests.

2.1 Proposal Requirements

All requests for an allocation are made via the LONI web interface to the LRAC. Start-up Allocations require only the data requested on the form. Large Allocations must attach a more extensive proposal in PDF format. The proposal is expected to be 5 pages or less, and concentrate on justifying the computational resources requested. The following outline should be followed:

1. Problem Statement - section limited to 1 page, or less, describing the desired outcomes of the project.
2. Background - section limited to 1 page or less, describing how the resources will be used to address the problem (i.e. student access for course work, specific models, etc.).
3. Methodology - section limited to 1 page, or less, describing the computational methodology that will be used. This should include the applications required.
4. Research Plan - section limited to 1 page, or less, describing the research schedule, including the anticipated expenditure of granted resources. Allocations are assumed to be uniformly consumed over their lifetime. If this will not be the case, an estimate of expenditure by calendar quarter is required.
5. Requirements Analysis - section limited to 2 pages, or less, detailing the basis for the requested computer time. Requests for Large Allocations must exhibit an understanding of application efficiency, scaling, and provide accurate estimations of the SU requirements.

Please note that section page limits are suggestions, but the maximum proposal page limit remains 5, not including any Attachments. The page limit was chosen with an eye to making it relatively easy to compose an allocation request, or to modify and reuse a successful application made to another center.

Project allocations are competitively reviewed and granted based upon the description of the proposed research and use of the available technology. They are not vetted on scientific merit. For Large Allocations, priority is given to funded research. All decisions made by the LRAC are deemed final. Appeals can be directed to the LMC or the member institution council representative.

2.2 Allocation Management

User accounts without access to a valid allocation will be blocked from submitting work on the systems. There is no mechanism for extending an allocation beyond one year, nor for adding resources once an allocation has been expended.

User accounts must be associated with a valid allocation, and if not, will be retained for a maximum of 1 year pending authorization against a renewed or different allocation. With these restrictions in mind, the PI is required to use the tools provided to monitor system usage and control authorization of project member accounts. PI's are strongly advised to carefully budget their usage appropriately throughout the year. PI's are ultimately responsible for assuring that a current and actively monitored management email address has been assigned to each allocation.

At the end of a Large Allocation, a short report must be submitted, as detailed above. Failure to submit this report may be used in the consideration of future allocations.

2.3 Early Allocation Access

If a PI who has already been awarded a Large Allocation by the LRAC puts in a new request, and there is a good reason to start the project before the next cycle, then the local representative can request staff to award as much as 25% of the project request immediately. Justification must be provided in the proposal. The LRAC committee must then be made aware of this action and its reasons by the local representative using the "LONI Allocations" listerver.

2.4 Reporting Requirement

At the end of a Large Allocation, a report in PDF format must be uploaded to the allocation web site. Requests for new Large Allocations will not be approved until reports for expired allocations have been submitted. The report (This template is provided as a reference) should detail:

1. If the goals of the proposed work were accomplished
2. Any major results or outcomes
3. All resulting grants/publications/presentations
4. Any research proposals submitted that made use of results generated.

3. Machine Access Policy

3.1 Job Queueing

Various workload balancing algorithms are used to determine how jobs are assigned resources on a given machine. The way a job is handled is determined by the job queue it is submitted to. Efficient use of the queuing system requires that users request runtimes consistent with estimated runtimes of their jobs. In particular, requesting more time than is necessary for a particular job can lead to inefficient and unfair queuing. Therefore, users that routinely request more time than is needed for their jobs are subject to a “priority penalty” that will lower the priority of their jobs. Each system sets a maximum number of jobs that a single user may have running without special permission (see below). There is no limit to the number of jobs that are particular use may have queued. Users that wish to obtain a higher priority for their jobs may use special priority queues (see below).

Descriptions of the different queues available on each cluster are provided in the User Guides. 

3.2. Queue Availability

The availability of the queues may be impacted by system maintenance and servicing. The appropriate job management commands can be used to check their status in real time. The maximum time allowed in the queues may be adjusted from time to time in order to improve overall utilization of the machines.

3.3. Storage

Disk space usage is controlled in one of 3 ways. The /home file system is quota controlled. A large pool of shared space is provided in the /work file system. Data in it is purged periodically, and users are cautioned to treat it as large, but temporary, storage space. Storage on the /project file system is quota controlled and may be requested via the allocation process. It serves as longer term (12 months at a time, subject to renewal and current demand) storage for very large data sets. It also supports user groups for sharing access to the data. For more details, see the separate LONI Storage Policy

3.4. Special Requests

A request for special access to LONI machines (such as usage of all nodes on a machine or exceptionally long runs) must be explicitly stated in the proposal for LONI resources. Appeals to the decision of the LRAC may be made to the LONI Management Council.

4. Resource Allocation Committee Members

The current LRAC members are shown in Table 4.

Each member holds approval authority for their respective institution. The LONI Director is also a member, and holds approval authority for the LONI Management Council.