Analyze and compute your large datasets

Large datasets may require special environments that can handle the storage and compute required to analyze the data. UCSF offers several secure options for analyzing and computing large datasets. Most of these options are paid services - it is important to budget appropriately early in your project.

Services 

  1. CoreHPC is UCSF’s next-generation high-performance computing (HPC) environment, designed to support large-scale data processing, AI-driven research, and compute-intensive scientific workloads. CoreHPC features a mix of GPU and CPU resources and is fully compliant with UC security policies and support P1 through P4 data classifications.
     
  2. Research Analysis Environment (RAE) offers windows Virtual Machine (VM) environments with no GPUs. However, a paid Premium option can provide additional compute with CPUs and RAM. The RAE team can work with you to "right size" your workloads and will adjust compute and storage specifications when needed. RAE can be used for P1 - P4 data.
     
  3. AWS Secure Enterprise Cloud (SEC) is a flexible cloud environment that can take on GPU and large compute cluster needs, and users pay for what they use. AWS SEC can be used for P1 - P4 data.

Directions

  1. Evaluate the different service options and determine the right fit for your project.
  2. Contact the service provider to obtain a budget and to get started.
  3. If you'd like to request a consultation to talk through these options, submit an IT Consultation Request.

Support

CoreHPC

Wynton

RAE

  • Use the RAE Help System to generate a support ticket.
  • Join the UCTech Slack channel #ucsf-rae to meet a community of peers and find the latest information about weekly office hours.
  • RAE users also have access to step-by-step documentation in the RAE support training library within MyTransfer.

AWS SEC

  • Submit an IT Consultation Request to assist you with budgeting for and configuring your environment.
  • AWS Solutions Architect Office Hours are also available every Wednesday at 11 a.m. Email [email protected] for the meeting link.