Работа с базами данных
<<  Коллекционирование 2 класс Мероприятия в очаге холеры схема  >>
“A Campus-Scale High Performance Cyberinfrastructure is Required for
“A Campus-Scale High Performance Cyberinfrastructure is Required for
Abstract
Abstract
Large Data Challenge: Average Throughput to End User on Shared
Large Data Challenge: Average Throughput to End User on Shared
OptIPuter Solution: Give Dedicated Optical Channels to Data-Intensive
OptIPuter Solution: Give Dedicated Optical Channels to Data-Intensive
The Global Lambda Integrated Facility-- Creating a Planetary-Scale
The Global Lambda Integrated Facility-- Creating a Planetary-Scale
Academic Research OptIPlanet Collaboratory: A 10Gbps “End-to-End”
Academic Research OptIPlanet Collaboratory: A 10Gbps “End-to-End”
The OptIPuter Project: Creating High Resolution Portals Over Dedicated
The OptIPuter Project: Creating High Resolution Portals Over Dedicated
MIT’s Ed DeLong and Darwin Project Team Using OptIPortal to Analyze
MIT’s Ed DeLong and Darwin Project Team Using OptIPortal to Analyze
AESOP Display built by Calit2 for KAUST-- King Abdullah University of
AESOP Display built by Calit2 for KAUST-- King Abdullah University of
The Latest OptIPuter Innovation: Quickly Deployable Nearly Seamless
The Latest OptIPuter Innovation: Quickly Deployable Nearly Seamless
The OctIPortable Being Checked Out Prior to Shipping to the
The OctIPortable Being Checked Out Prior to Shipping to the
3D Stereo Head Tracked OptIPortal: NexCAVE
3D Stereo Head Tracked OptIPortal: NexCAVE
High Definition Video Connected OptIPortals: Virtual Working Spaces
High Definition Video Connected OptIPortals: Virtual Working Spaces
“Blueprint for the Digital University”--Report of the UCSD Research
“Blueprint for the Digital University”--Report of the UCSD Research
Calit2 Sunlight OptIPuter Exchange Connects 60 Campus Sites Each
Calit2 Sunlight OptIPuter Exchange Connects 60 Campus Sites Each
UCSD Campus Investment in Fiber Enables Consolidation of Energy
UCSD Campus Investment in Fiber Enables Consolidation of Energy
Data-Intensive Supercomputer Based on SSD Flash Memory and Virtual
Data-Intensive Supercomputer Based on SSD Flash Memory and Virtual
Gordon Bests Previous Mega I/O per Second by 25x
Gordon Bests Previous Mega I/O per Second by 25x
Rapid Evolution of 10GbE Port Prices Makes Campus-Scale 10Gbps CI
Rapid Evolution of 10GbE Port Prices Makes Campus-Scale 10Gbps CI
Arista Enables SDSC’s Massive Parallel 10G Switched Data Analysis
Arista Enables SDSC’s Massive Parallel 10G Switched Data Analysis
The Next Step for Data-Intensive Science: Pioneering the HPC Cloud
The Next Step for Data-Intensive Science: Pioneering the HPC Cloud
Data Oasis – 3 Different Types of Storage
Data Oasis – 3 Different Types of Storage
Examples of Applications Built on UCSD RCI
Examples of Applications Built on UCSD RCI
Exploring Cosmology With Supercomputers, Supernetworks, and
Exploring Cosmology With Supercomputers, Supernetworks, and
Providing End-to-End CI for Petascale End Users
Providing End-to-End CI for Petascale End Users
Using Supernetworks to Couple End User’s OptIPortal to Remote
Using Supernetworks to Couple End User’s OptIPortal to Remote
Most of Evolutionary Time Was in the Microbial World
Most of Evolutionary Time Was in the Microbial World
The New Science of Microbial Metagenomics
The New Science of Microbial Metagenomics
Calit2 Microbial Metagenomics Cluster- Next Generation Optically
Calit2 Microbial Metagenomics Cluster- Next Generation Optically
Calit2 CAMERA: Over 4000 Registered Users From Over 80 Countries
Calit2 CAMERA: Over 4000 Registered Users From Over 80 Countries
Creating CAMERA 2.0 - Advanced Cyberinfrastructure Service Oriented
Creating CAMERA 2.0 - Advanced Cyberinfrastructure Service Oriented
The GreenLight Project: Instrumenting the Energy Cost of Computational
The GreenLight Project: Instrumenting the Energy Cost of Computational
GreenLight Project: Remote Visualization of Data Center
GreenLight Project: Remote Visualization of Data Center
GreenLight Projects Airflow dynamics
GreenLight Projects Airflow dynamics
GreenLight Project Heat Distribution
GreenLight Project Heat Distribution
Cost Per Megabase in Sequencing DNA is Falling Much Faster Than
Cost Per Megabase in Sequencing DNA is Falling Much Faster Than
BGI—The Beijing Genome Institute is the World’s Largest Genomic
BGI—The Beijing Genome Institute is the World’s Largest Genomic
From 10,000 Human Genomes Sequenced in 2011 to 1 Million by 2015 in
From 10,000 Human Genomes Sequenced in 2011 to 1 Million by 2015 in
Needed: Interdisciplinary Teams Made From Computer Science, Data
Needed: Interdisciplinary Teams Made From Computer Science, Data
Calit2 Brings Together Computer Science and Bioinformatics
Calit2 Brings Together Computer Science and Bioinformatics
GreenLight Project Allows for Testing of Novel Architectures on
GreenLight Project Allows for Testing of Novel Architectures on
Using UCSD RCI to Store and Analyze Next Gen Sequencer Datasets
Using UCSD RCI to Store and Analyze Next Gen Sequencer Datasets
NIH National Center for Microscopy & Imaging Research Integrated
NIH National Center for Microscopy & Imaging Research Integrated
UCSD Planned Optical Networked Biomedical Researchers and Instruments
UCSD Planned Optical Networked Biomedical Researchers and Instruments

Презентация на тему: «A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research». Автор: Jerry Sheehan. Файл: «A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research.ppt». Размер zip-архива: 10774 КБ.

A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research

содержание презентации «A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research.ppt»
СлайдТекст
1 “A Campus-Scale High Performance Cyberinfrastructure is Required for

“A Campus-Scale High Performance Cyberinfrastructure is Required for

Data-Intensive Research”

Seminar Presentation Princeton Institute for Computational Science and Engineering (PICSciE) Princeton University Princeton, NJ December 12, 2011

Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD http://lsmarr.calit2.net

1

2 Abstract

Abstract

Campuses are experiencing an enormous increase in the quantity of data generated by scientific instruments and computational clusters and stored in massive data repositories. The shared Internet, engineered to enable interaction with megabyte-sized data objects is not capable of dealing with the typical gigabytes to terabytes of modern scientific data. Instead, a high performance cyberinfrastructure is emerging to support data-intensive research. Fortunately, multi-channel optical fiber can support both the traditional internet and this new data utility. I will give examples of early prototypes which integrate data generation, transmission, storage, analysis, visualization, curation, and sharing, driven by applications as diverse as genomics, ocean observatories, and cosmology.

3 Large Data Challenge: Average Throughput to End User on Shared

Large Data Challenge: Average Throughput to End User on Shared

Internet is 10-100 Mbps

Transferring 1 TB: --50 Mbps = 2 Days --10 Gbps = 15 Minutes

http://ensight.eos.nasa.gov/Missions/terra/index.shtml

Tested December 2011

4 OptIPuter Solution: Give Dedicated Optical Channels to Data-Intensive

OptIPuter Solution: Give Dedicated Optical Channels to Data-Intensive

Users

10 Gbps per User ~ 100x Shared Internet Throughput

Parallel Lambdas are Driving Optical Networking The Way Parallel Processors Drove 1990s Computing

5 The Global Lambda Integrated Facility-- Creating a Planetary-Scale

The Global Lambda Integrated Facility-- Creating a Planetary-Scale

High Bandwidth Collaboratory

Research Innovation Labs Linked by 10G Dedicated Lambdas

www.glif.is/publications/maps/GLIF_5-11_World_2k.jpg

6 Academic Research OptIPlanet Collaboratory: A 10Gbps “End-to-End”

Academic Research OptIPlanet Collaboratory: A 10Gbps “End-to-End”

Lightpath Cloud

HD/4k Live Video

HPC

Local or Remote Instruments

End User OptIPortal

National LambdaRail

Campus Optical Switch

Data Repositories & Clusters

HD/4k Video Repositories

10G Lightpaths

7 The OptIPuter Project: Creating High Resolution Portals Over Dedicated

The OptIPuter Project: Creating High Resolution Portals Over Dedicated

Optical Channels to Global Science Data

OptIPortal

Scalable Adaptive Graphics Environment (SAGE)

Calit2 (UCSD, UCI), SDSC, and UIC Leads—Larry Smarr PI Univ. Partners: NCSA, USC, SDSU, NW, TA&M, UvA, SARA, KISTI, AIST Industry: IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent

Picture Source: Mark Ellisman, David Lee, Jason Leigh

8 MIT’s Ed DeLong and Darwin Project Team Using OptIPortal to Analyze

MIT’s Ed DeLong and Darwin Project Team Using OptIPortal to Analyze

10km Ocean Microbial Simulation

Cross-Disciplinary Research at MIT, Connecting Systems Biology, Microbial Ecology, Global Biogeochemical Cycles and Climate

9 AESOP Display built by Calit2 for KAUST-- King Abdullah University of

AESOP Display built by Calit2 for KAUST-- King Abdullah University of

Science & Technology

40-Tile 46” Diagonal Narrow-Bezel AESOP Display at KAUST Running CGLX

10 The Latest OptIPuter Innovation: Quickly Deployable Nearly Seamless

The Latest OptIPuter Innovation: Quickly Deployable Nearly Seamless

OptIPortables

Shipping Case

Image From the Calit2 KAUST Lab

45 minute setup, 15 minute tear-down with two people (possible with one)

11 The OctIPortable Being Checked Out Prior to Shipping to the

The OctIPortable Being Checked Out Prior to Shipping to the

Calit2/KAUST Booth at SIGGRAPH 2011

Photo:Tom DeFanti

12 3D Stereo Head Tracked OptIPortal: NexCAVE

3D Stereo Head Tracked OptIPortal: NexCAVE

Array of JVC HDTV 3D LCD Screens KAUST NexCAVE = 22.5MPixels

www.calit2.net/newsroom/article.php?id=1584

Source: Tom DeFanti, Calit2@UCSD

13 High Definition Video Connected OptIPortals: Virtual Working Spaces

High Definition Video Connected OptIPortals: Virtual Working Spaces

for Data Intensive Research

2010

NASA Supports Two Virtual Institutes

LifeSize HD

Calit2@UCSD 10Gbps Link to NASA Ames Lunar Science Institute, Mountain View, CA

Source: Falko Kuester, Kai Doerr Calit2; Michael Sims, Larry Edwards, Estelle Dodson NASA

14 “Blueprint for the Digital University”--Report of the UCSD Research

“Blueprint for the Digital University”--Report of the UCSD Research

Cyberinfrastructure Design Team

No Data Bottlenecks--Design for Gigabit/s Data Flows

A Five Year Process Begins Pilot Deployment This Year

April 2009

research.ucsd.edu/documents/rcidt/RCIDTReportFinal2009.pdf

15 Calit2 Sunlight OptIPuter Exchange Connects 60 Campus Sites Each

Calit2 Sunlight OptIPuter Exchange Connects 60 Campus Sites Each

Dedicated at 10Gbps

Maxine Brown, EVL, UIC OptIPuter Project Manager

16 UCSD Campus Investment in Fiber Enables Consolidation of Energy

UCSD Campus Investment in Fiber Enables Consolidation of Energy

Efficient Computing & Storage

Source: Philip Papadopoulos, SDSC, UCSD

17 Data-Intensive Supercomputer Based on SSD Flash Memory and Virtual

Data-Intensive Supercomputer Based on SSD Flash Memory and Virtual

Shared Memory SW Emphasizes MEM and IOPS over FLOPS Supernode has Virtual Shared Memory: 2 TB RAM Aggregate 8 TB SSD Aggregate Total Machine = 32 Supernodes 4 PB Disk Parallel File System >100 GB/s I/O System Designed to Accelerate Access to Massive Datasets being Generated in Many Fields of Science, Engineering, Medicine, and Social Science

NSF Funds a Big Data Supercomputer: SDSC’s Gordon-Dedicated Dec. 5, 2011

Source: Mike Norman, Allan Snavely SDSC

18 Gordon Bests Previous Mega I/O per Second by 25x

Gordon Bests Previous Mega I/O per Second by 25x

19 Rapid Evolution of 10GbE Port Prices Makes Campus-Scale 10Gbps CI

Rapid Evolution of 10GbE Port Prices Makes Campus-Scale 10Gbps CI

Affordable

Port Pricing is Falling Density is Rising – Dramatically Cost of 10GbE Approaching Cluster HPC Interconnects

Source: Philip Papadopoulos, SDSC/Calit2

$80K/port Chiaro (60 Max)

$ 5K Force 10 (40 max)

~$1000 (300+ Max)

$ 500 Arista 48 ports

$ 400 Arista 48 ports

2005 2007 2009 2010

20 Arista Enables SDSC’s Massive Parallel 10G Switched Data Analysis

Arista Enables SDSC’s Massive Parallel 10G Switched Data Analysis

Resource

Triton

Trestles 100 TF

Dash

Gordon

10Gbps

Radical Change Enabled by Arista 7508 10G Switch 384 10G Capable

Existing Commodity Storage 1/3 PB

Phase0: > 8GB/s Sustained Today Phase I: > 50 GB/sec for Lustre (May 2011) :Phase II: >100 GB/s (Feb 2012)

Source: Philip Papadopoulos, SDSC/Calit2

2000 TB > 50 GB/s

UCSD RCI

OptIPuter

Co-Lo

CENIC/NLR

Oasis Procurement (RFP)

5

8

2

32

4

8

32

2

12

40?128

8

128

21 The Next Step for Data-Intensive Science: Pioneering the HPC Cloud

The Next Step for Data-Intensive Science: Pioneering the HPC Cloud

22 Data Oasis – 3 Different Types of Storage

Data Oasis – 3 Different Types of Storage

23 Examples of Applications Built on UCSD RCI

Examples of Applications Built on UCSD RCI

DOE Remote Use of Petascale HPC Moore Foundation Microbial Metagenomics Server NSF GreenLight Instrumented Data Center NIH Next Generation Gene Sequencers NIH Shared Scientific Instruments

24 Exploring Cosmology With Supercomputers, Supernetworks, and

Exploring Cosmology With Supercomputers, Supernetworks, and

Supervisualization

40963 Particle/Cell Hydrodynamic Cosmology Simulation NICS Kraken (XT5) 16,384 cores Output 148 TB Movie Output (0.25 TB/file) 80 TB Diagnostic Dumps (8 TB/file)

Intergalactic Medium on 2 GLyr Scale

Science: Norman, Harkness,Paschos SDSC Visualization: Insley, ANL; Wagner SDSC

Source: Mike Norman, SDSC

ANL * Calit2 * LBNL * NICS * ORNL * SDSC

25 Providing End-to-End CI for Petascale End Users

Providing End-to-End CI for Petascale End Users

log of gas temperature log of gas density

Two 64K Images From a Cosmological Simulation of Galaxy Cluster Formation

Mike Norman, SDSC October 10, 2008

26 Using Supernetworks to Couple End User’s OptIPortal to Remote

Using Supernetworks to Couple End User’s OptIPortal to Remote

Supercomputers and Visualization Servers

Source: Mike Norman, Rick Wagner, SDSC

Real-Time Interactive Volume Rendering Streamed from ANL to SDSC

*ANL * Calit2 * LBNL * NICS * ORNL * SDSC

27 Most of Evolutionary Time Was in the Microbial World

Most of Evolutionary Time Was in the Microbial World

Earth is a Microbial World: For Every Human Cell There are 100 Million Microbes

Tree of Life Derived from 16S rRNA Sequences

Source: Carl Woese, et al

28 The New Science of Microbial Metagenomics

The New Science of Microbial Metagenomics

NRC Report: Metagenomic data should be made publicly available in international archives as rapidly as possible.

“The emerging field of metagenomics, where the DNA of entire communities of microbes is studied simultaneously, presents the greatest opportunity – perhaps since the invention of the microscope – to revolutionize understanding of the microbial world.” – National Research Council March 27, 2007

29 Calit2 Microbial Metagenomics Cluster- Next Generation Optically

Calit2 Microbial Metagenomics Cluster- Next Generation Optically

Linked Science Data Server

Grant Announced January 17, 2006

30 Calit2 CAMERA: Over 4000 Registered Users From Over 80 Countries

Calit2 CAMERA: Over 4000 Registered Users From Over 80 Countries

http://camera.calit2.net/

Community Cyberinfrastructure for Advanced Microbial Ecology Research and Analysis

31 Creating CAMERA 2.0 - Advanced Cyberinfrastructure Service Oriented

Creating CAMERA 2.0 - Advanced Cyberinfrastructure Service Oriented

Architecture

Source: CAMERA CTO Mark Ellisman

32 The GreenLight Project: Instrumenting the Energy Cost of Computational

The GreenLight Project: Instrumenting the Energy Cost of Computational

Science

Focus on 5 Communities with At-Scale Computing Needs: Metagenomics Ocean Observing Microscopy Bioinformatics Digital Media Measure, Monitor, & Web Publish Real-Time Sensor Outputs Via Service-oriented Architectures Allow Researchers Anywhere To Study Computing Energy Cost Enable Scientists To Explore Tactics For Maximizing Work/Watt Develop Middleware that Automates Optimal Choice of Compute/RAM Power Strategies for Desired Greenness Data Center for School of Medicine Illumina Next Gen Sequencer Storage and Processing

Source: Tom DeFanti, Calit2; GreenLight PI

33 GreenLight Project: Remote Visualization of Data Center

GreenLight Project: Remote Visualization of Data Center

34 GreenLight Projects Airflow dynamics

GreenLight Projects Airflow dynamics

Live fan speeds

Airflow dynamics

34

35 GreenLight Project Heat Distribution

GreenLight Project Heat Distribution

Combined heat + fans

Realistic correlation

36 Cost Per Megabase in Sequencing DNA is Falling Much Faster Than

Cost Per Megabase in Sequencing DNA is Falling Much Faster Than

Moore’s Law

www.genome.gov/sequencingcosts/

37 BGI—The Beijing Genome Institute is the World’s Largest Genomic

BGI—The Beijing Genome Institute is the World’s Largest Genomic

Institute

Main Facilities in Shenzhen and Hong Kong, China Branch Facilities in Copenhagen, Boston, UC Davis 137 Illumina HiSeq 2000 Next Generation Sequencing Systems Each Illumina Next Gen Sequencer Generates 25 Gigabases/Day Supported by High Performance Computing and Storage ~160TF, 33TB Memory Large-Scale (12PB) Storage

38 From 10,000 Human Genomes Sequenced in 2011 to 1 Million by 2015 in

From 10,000 Human Genomes Sequenced in 2011 to 1 Million by 2015 in

Less Than 5,000 sq. ft.!

4 Million Newborns / Year in U.S.

39 Needed: Interdisciplinary Teams Made From Computer Science, Data

Needed: Interdisciplinary Teams Made From Computer Science, Data

Analytics, and Genomics

40 Calit2 Brings Together Computer Science and Bioinformatics

Calit2 Brings Together Computer Science and Bioinformatics

41 GreenLight Project Allows for Testing of Novel Architectures on

GreenLight Project Allows for Testing of Novel Architectures on

Bioinformatics Algorithms

“Our version of MS-Alignment [a proteomics algorithm] is more than 115x faster than a single core of an Intel Nehalem processor, is more than 15x faster than an eight-core version, and reduces the runtime for a few samples from 24 hours to just a few hours.” —From “Computational Mass Spectrometry in a Reconfigurable Coherent Co-processing Architecture,” IEEE Design & Test of Computers, Yalamarthy (ECE), Coburn (CSE), Gupta (CSE), Edwards (Convey), and Kelly (Convey) (2011)

June 23, 2009

http://research.microsoft.com/en-us/um/cambridge/events/date2011/msalignment_dateposter_2011.pdf

42 Using UCSD RCI to Store and Analyze Next Gen Sequencer Datasets

Using UCSD RCI to Store and Analyze Next Gen Sequencer Datasets

Stream Data from Genomics Lab to GreenLight Storage, NFS Mount Over 10Gbps to Triton Compute Cluster

Source: Chris Misleh, SOM/Calit2 UCSD

43 NIH National Center for Microscopy & Imaging Research Integrated

NIH National Center for Microscopy & Imaging Research Integrated

Infrastructure of Shared Resources

Shared Infrastructure

Scientific Instruments

Local SOM Infrastructure

End User Workstations

Source: Steve Peltier, Mark Ellisman, NCMIR

44 UCSD Planned Optical Networked Biomedical Researchers and Instruments

UCSD Planned Optical Networked Biomedical Researchers and Instruments

Connects at 10 Gbps : Microarrays Genome Sequencers Mass Spectrometry Light and Electron Microscopes Whole Body Imagers Computing Storage

«A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research»
http://900igr.net/prezentacija/informatika/a-campus-scale-high-performance-cyberinfrastructure-is-required-for-data-intensive-research-241340.html
cсылка на страницу

Работа с базами данных

11 презентаций о работе с базами данных
Урок

Информатика

130 тем
Слайды
900igr.net > Презентации по информатике > Работа с базами данных > A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research