<<  Poggio Antico A combination of people, love and passion for winemaking in the beautiful landscape of Tuscany The future perspectives of applications of artificial neural networks for diagnostics, modeling, and control in science and industry (methods, results, demonstrations hands-on)  >>
The IBM view on storage archive solutions: requirements to solve and
The IBM view on storage archive solutions: requirements to solve and
Agenda
Agenda
Growth Digital Universe 2011 1.800 ExaByte (1,8 billionsTB)
Growth Digital Universe 2011 1.800 ExaByte (1,8 billionsTB)
Every day 15 PetaBytes of new information in digital format are
Every day 15 PetaBytes of new information in digital format are
Smarter Systems Are Creating an Information Explosion
Smarter Systems Are Creating an Information Explosion
Changing Market Dynamics & Trends
Changing Market Dynamics & Trends
Significant growth expected in Digital Archiving
Significant growth expected in Digital Archiving
Why store data for long-term, and how
Why store data for long-term, and how
What to archive and how much time
What to archive and how much time
You Might Think Archiving Means any of These
You Might Think Archiving Means any of These
Major Archive Segments
Major Archive Segments
Technologies for data archiving and preservation
Technologies for data archiving and preservation
Storage management at 360: archiving, backup, migration, DR
Storage management at 360: archiving, backup, migration, DR
The IBM Smart Archive strategy
The IBM Smart Archive strategy
Long term data archiving: Total Cost
Long term data archiving: Total Cost
Long term data archiving: TCO and technology evolution
Long term data archiving: TCO and technology evolution
Tape Advantages for Archiving/Long-Term Preservation
Tape Advantages for Archiving/Long-Term Preservation
Technology Roadmap Comparisons for TAPE, HDD, and NAND Flash Outline :
Technology Roadmap Comparisons for TAPE, HDD, and NAND Flash Outline :
Annual Areal Density Growth Rate Scenarios
Annual Areal Density Growth Rate Scenarios
Cost evolution of the magnetic storage
Cost evolution of the magnetic storage
Magnetic Tape
Magnetic Tape
Rich Media Driving New Storage Requirements
Rich Media Driving New Storage Requirements
Elements to address new role of TAPE
Elements to address new role of TAPE
LTFS Value Proposition
LTFS Value Proposition
Introduction to LTFS (Linear Tape File System)
Introduction to LTFS (Linear Tape File System)
Logical View of LTFS Volume
Logical View of LTFS Volume
IBM : 60 Years of Tape Innovation
IBM : 60 Years of Tape Innovation
LTO Roadmap
LTO Roadmap
And data deduplication is the key to using more disk more cost
And data deduplication is the key to using more disk more cost
IBM ProtecTIER Deduplication Family
IBM ProtecTIER Deduplication Family
Koninklijke Bibliotheek National Library of the Netherlands
Koninklijke Bibliotheek National Library of the Netherlands
Ecosystem: Thought Equity Motion Sports Video Archiving in the Cloud
Ecosystem: Thought Equity Motion Sports Video Archiving in the Cloud
Thank you for your attention
Thank you for your attention

: The IBM view on storage archive solutions: requirements to solve and trends for the future 31st. : mcbarton. : The IBM view on storage archive solutions: requirements to solve and trends for the future 31st.ppt. zip-: 6379 .

The IBM view on storage archive solutions: requirements to solve and trends for the future 31st

The IBM view on storage archive solutions: requirements to solve and trends for the future 31st.ppt
1 The IBM view on storage archive solutions: requirements to solve and

The IBM view on storage archive solutions: requirements to solve and

trends for the future 31st ADLUG ANNUAL MEETING - Firenze, September 19-21st

IBM Systems and Technology Group

Marco Ceresoli Data Protection and Retention Sales Leader IBM Europe

2 Agenda

Agenda

The growth and the variety of digital information The shift of market dynamics and trends for Archiving Technologies for data archiving: comparison New trends: Linear Tape File System value proposition Role and history of IBM in Tape technology Case studies and conclusions

3 Growth Digital Universe 2011 1.800 ExaByte (1,8 billionsTB)

Growth Digital Universe 2011 1.800 ExaByte (1,8 billionsTB)

Growth Digital Universe 2005 150 ExaByte (150 millions TB)

Storage is growing and not only in terms of capacity

Velocity

Variety

Volumes

Source: 2011 IDC Digital Universe Study

4 Every day 15 PetaBytes of new information in digital format are

Every day 15 PetaBytes of new information in digital format are

created 80% of this new data is unstructured generated mainly by email, documents, images, video and audio. EFFECTS A company with 1,000 employees spend on average 5,3M$ every year to search for information which is difficult to find. 42% of managers say that they utilize INCORRECT information at least once a week. During 2007 in the USA there were 37.000 security breaches (cyber attacks) with an increment of 158% versus 2006. More than 20.000 laws at global level require not only pure storage capacity but classification and Information lifecycle management.

Information Week, State Of Enterprise Storage Changing Priorities, Changing Practices, 2009.

5 Smarter Systems Are Creating an Information Explosion

Smarter Systems Are Creating an Information Explosion

Source:: Semantics, Linked Data guidelines, 2006.

6 Changing Market Dynamics & Trends

Changing Market Dynamics & Trends

Value has Shifted toward Archiving Software Shift from Hardware to Archiving Software for addressing compliance, data retention management and lifecycle governance requirements Email archiving and eDiscovery adding additional content types Information Lifecycle Governance is needed Clients understand they can no longer address data growth issues by adding more storage Backup as Archive Significant proportion (over 50%) of customers continue to use backups as archive copies for long term retention Industry Specific Archives Healthcare & Life Sciences requirements for archival of Medical Images and Electronic Medical Records Government, Oil & Gas, and other industries demanding solutions specific to their needs Cross-Industry requirements also rising (e.g., Compliance, retaining Surveillance data for long periods of time) Cloud Based Archiving Hosted offerings replaced by clouds (e.g., for eDiscovery) Shift in deployment models from siloed on-premise installations to consolidated solutions, archive as a service, and cloud archiving

7 Significant growth expected in Digital Archiving

Significant growth expected in Digital Archiving

Archival (Tier 3) data is: Fastest growing at 65% CAGR Stored on Disk, Tape, and Optical Media (Not captured in Tape IDC or GMV forecasts)

Graph illustrates Active and Deep Archiving combined

8 Why store data for long-term, and how

Why store data for long-term, and how

Why I need to store for a long time? Cultural and scientific vale Value for the company More than 22.000 norms/laws at worldwide level to rule the data preservation

How to store this data? Multi-level storage infrastructure with different costs Data reduction (compression and data deduplication) Automatic data management based on archiving rules Virtualization and independence from the storage infrastructure anywhere and self-service accessibility cloud-oriented Focus on storing documents and data interconnections (metadata) together

9 What to archive and how much time

What to archive and how much time

Which data needs to be stored?

How long to store?

Source: ESG - Requested Record Types During Electronic Discovery Processes

Source: SNIA 100 Year Archive Requirements Survey

10 You Might Think Archiving Means any of These

You Might Think Archiving Means any of These

Archive -- a long-term collection of data that typically is fixed-content data; i.e., no I/O writes are allowed to change the data. Deep archiving The original definition of archiving, whereby production data is written to another set of storage media (typically tape) and moved offsite while the original version is deleted (typically from disk). Active archiving Data for which frequency of access is active rather than inactive, while frequency of updating is nonexistent so the data is fixed (i.e., is unchanging) and not subject to I/O writes that could change the data. Long-term archiving Active archived data for which the frequency of access has fallen so low that a tier of more cost-effective storage may be an appropriate place to house the data. Backup a dated (i.e., specified-time) duplication of a designated set of data from a data source on one set of media (typically disk) to a backup set of media (either disk or tape) Vaulting Typically, the movement of data on tapes from a target site to a protected remote site. Source of these definitions: Data Protection, David Hill, 2009, CRC Press

11 Major Archive Segments

Major Archive Segments

Structured Data (database archiving) What? Relational tables, rows, periodic reports, retire applications Why? Reduce storage growth, improve performance, lower cost, Compliance (reports) Available products? IBM Optim with IBM disk storage

eMail archiving, eDiscovery . What? email, but any other data type potentially too Why? Litigation support, Compliance Available products? IBM Content Collector with IBM disk storage

Unstructured Data (files) What? MS office, SharePoint, contracts, images, etc. Why? Reduce storage growth, offer a service or product, improve performance, lower cost, Compliance Available products? IBM Content Collector, FileNet, Content Manager, etc.

Unstructured Data (kept from birth) What? Medical Images, Content (M&E), DVS, Seismic shots, Scientific Why? Reduce storage growth, offer a service or product, improve performance, lower cost Available products? VAD Medical Archive solution or or LTFS/tape with an ISV app

12 Technologies for data archiving and preservation

Technologies for data archiving and preservation

More than 50 years of continuous innovation

Fault tolerance: redundancy, ECC, RAID(*), ... Data protection: space-efficient internal replication Disaster recovery: automated remote data replication Data immutability: NENR(*) e WORM(*) Archiving and preservation rules: API(*) and standard interfaces Cost reduction: storage tiering, WORN(*) Data growth reduction: data deduplication and data compression Data security: data encryption and data shredding Access control: tamper protection, audit logs, ...

(*) ECC = Error Correction Code, RAID = Redundant Array of Independent Disk, NENR = Non Erasable Non Rewritable, WORM = Write Once Read Many, API = Application Program Interface, WORN = Write Once Read Never

13 Storage management at 360: archiving, backup, migration, DR

Storage management at 360: archiving, backup, migration, DR

The processes can be automated and repeated

backup copies

Migration to new technologies

Archiving and ILM management

Disaster protection

NENR/WORM storage

Compression? De-duplication?

Encryption?

Automated Off-line

Manual Off-line

Mid-range

Low-cost

Enterprise class

WORM

NENR

NENR

14 The IBM Smart Archive strategy

The IBM Smart Archive strategy

Optimized and Unified Assessment, Collection and Classification

Value Added Services Optimization Services System Services Managed Services Reference Architecture Information Governance

Flexible and Secure Infrastructure with Unified Retention and Protection

Cloud Ready Archive Storage with Optional ECM

Integrated Compliance, Records Management, Analytics and eDiscovery

15 Long term data archiving: Total Cost

Long term data archiving: Total Cost

From: In Search of the Long-Term Archiving Solution - Tape Delivers Significant TCO Advantage over Disk, The Clipper Group, Dec.23, 2010.

16 Long term data archiving: TCO and technology evolution

Long term data archiving: TCO and technology evolution

From: In Search of the Long-Term Archiving Solution - Tape Delivers Significant TCO Advantage over Disk, The Clipper Group, Dec.23, 2010.

17 Tape Advantages for Archiving/Long-Term Preservation

Tape Advantages for Archiving/Long-Term Preservation

Disk

Tape

Source: Tape The Digital Curator of the Information Age. By Fred Moore, President, Horison, Inc.

18 Technology Roadmap Comparisons for TAPE, HDD, and NAND Flash Outline :

Technology Roadmap Comparisons for TAPE, HDD, and NAND Flash Outline :

Implications for Data Storage Applications

The annual rate of areal density increases for TAPE will likely exceed the annual rate of areal density increases for NAND and HDD TAPE bit cell is large and paths for scaling to higher bit densities exist NAND bit cells and HDD Patterned Media bit cells are approaching nanoscale issues in minimum feature lithography requirements NAND bit endurance or bit retention and HDD bit stability are approaching A Possible Annual Areal Density Growth Scenarios 20% for HDD 20% to 30% for NAND Flash 40% to 80% for TAPE Implications for Storage: TAPE, NAND, and HDD will continue to offer complementary storage solutions Implications for TAPE: TAPE volumetric density will increase, enhancing its cost advantages

19 Annual Areal Density Growth Rate Scenarios

Annual Areal Density Growth Rate Scenarios

AREAL DENSITY (Gbit/in?)

YEAR

10000

1000

100

10

1

0.1

2002

2004

2006

2008

2010

2012

2014

2016

2018

HDD 20% to 25% Transition to New Technology, Sensor Output, Lithography NAND Flash 25% to 30% Lithography and Endurance TAPE 40% to 80% -- No Lithography Issues, Mechanical Realities

20 Cost evolution of the magnetic storage

Cost evolution of the magnetic storage

~6-10X

SSD

Source: IBM elaboration and Information Storage Industry Consortium (INSIC) 2008

21 Magnetic Tape

Magnetic Tape

The cheaper storage support of the hierarchy Most used for long term archiving purposes LTO (Linear Tape Open) standard: Fifth generation available today with 1,5TB cartridges (3TB compressed) January 2010: the IBM Zurich Research Laboratory performed a technology demonstration of a 35TB cartridge(1) . Today they are working on a technology demo of a 100TB cartridge.

http://lto.org/technology/roadmap.html

(1) http://www.ibm.com/press/us/en/pressrelease/29245.wss

22 Rich Media Driving New Storage Requirements

Rich Media Driving New Storage Requirements

Characteristics of data stored is changing Mix of traditional business data (ie. transactional, docs, email, databases, and backup of those assets) vs rich media (ie. video, images, digitized content, etc) is rapidly changing

Access & asset management profiles of rich media are significantly different from traditional business data Much of traditional business data stored is a cost center Regulatory, compliance, disaster recovery for business critical data and processes Rich media is primarily stored for monetization purposes Production archives and asset protection Repurposing content and distribution Long term archives to monetize assets BW changes everything access to/from content, business motivation to make content available

Video, images, etc. a major factor driving growth

Eg. Key to M&E industry move to digital workflows

23 Elements to address new role of TAPE

Elements to address new role of TAPE

Self-Describing cartridge Remove requirement to commit long term to tape software application Content protection in event of database corruption or loss Improve content interchange/distribution Eliminate need for common tape software across enterprise and/or interchange locations Reduce cost of data interchange Partial Recall Eliminate time penalty in moving large video content to tape in event of need small part of video content (ie. Goal in game) Improved Tier management of content Ease complexity in movement from Tier 1 (disk) to Tier 2 (online tape) and Tier 3 (archive) Improve data import/export to system management $/GB, Power Reduce cost of digital storage power and $/min Open Standards Large diverse infrastructure requires open standard Standard/support of MXF video Long Term Content Archive Life Archive life desire for 50-100 years

24 LTFS Value Proposition

LTFS Value Proposition

Digital archives need and want the Value Proposition of Tape: $/GB lowest cost storage Watt/GB green storage Portability ability to manage archive outside system Scalability easy to add additional storage (ie. buy cartridge) Investment protection LTO has an 8 generation roadmap (up to a 32TB cartridge (compr.)) But - Inhibitors to use tape: Proprietary tape applications require long term commitment and support of tape application to maintain archive Non-self describing data formats requiring centralized archive database to recover content on individual tapes Import/export & distribution of tapes in archive is difficult due to proprietary tape applications Solution: LTFS addresses the inhibitors and unlocks value proposition of tape for digital archives Open, non-proprietary tape format Self-describing data structure on cartridge File system support on Linux, Mac, Windows provides: Distribution and cross platform interchange Enables transition to integrated file based tape/disk storage systems

25 Introduction to LTFS (Linear Tape File System)

Introduction to LTFS (Linear Tape File System)

IBM Linear Tape File System is: Open Format for data which is written to tape Describes the format of data and meta data stored on tape Meta data is based on XML schema Developed and disclosed by IBM Applicable to LTO-5 and Jag-4 Requires tape partitioning File System support (code) to R/W tapes in LTFS format externalizes the LTO-5 tape as file system Enables standard applications to write/read LTFS tapes Supports update, edit, delete of files on LTFS tape Supports partial recall Available on Linux, Mac OS X and Windows

26 Logical View of LTFS Volume

Logical View of LTFS Volume

Index Partition

Data Partition

Guard Wraps

LTFS utilizes media partitioning (new to LTO Gen 5 and Jag 4) The tape is logically divided lengthwise (think C: & D: drives on single hard disk unit) LTFS places the index on one partition and data on the other

27 IBM : 60 Years of Tape Innovation

IBM : 60 Years of Tape Innovation

In tape drive technology

In tape automation and virtualization

(3592 G4)

2011

TS3500

Connector & Shuttle

1952

1952

1964

1964

1995

1995

IBM 726

IBM 726

IBM 2104

IBM 2104

2000

2000

2002

2002

2004

2004

2007

2007

2010

1

1

magnetic tape drive

magnetic tape drive

1

1

read/back drive

read/back drive

IBM 3590

IBM 3590

LTO Gen1

LTO Gen1

LTO Gen2

LTO Gen2

LTO Gen3

LTO Gen3

LTO Gen4

LTO Gen4

LTO Gen5

1959

1959

1984

1984

1999

1999

2003

2003

2005

2005

2008

2008

2011

IBM 729

IBM 729

IBM 3480

IBM 3480

IBM 3590E

IBM 3590E

3592 Gen1

3592 Gen1

TS1120

TS1120

TS1130

TS1130

TS1140

1

1

read/write drive

read/write drive

1

1

cartridge drive

cartridge drive

(3592 G2)

(3592 G2)

(3592 G3)

(3592 G3)

2008

2008

1992

1992

TS2900

TS2900

IBM 3495

IBM 3495

2005

2005

1962

TS3500

TS3500

TS3200

TS3200

2000

2000

2007

2007

1994

1994

IBM Tractor System

High Density

High Density

TS3300

TS3300

TS3400

TS3400

TS3500

TS3500

IBM 3494

IBM 3494

2008

2008

1999

1999

1997

1997

2001

2001

2006

2006

1974

TS7720

TS7720

VTS G2

VTS G2

VTS G1

VTS G1

VTS G3

VTS G3

TS7740 (VTS Gen 4)

TS7740 (VTS Gen 4)

3850 MSS

2009

2009

2005

2005

2007

2007

2008

2008

2007

2007

TS7650

TS7650

TS7510 VTL

TS7510 VTL

TS7520

TS7520

TS7650G

TS7650G

TS7530

TS7530

Appliance

Appliance

st

st

st

st

st

st

st

st

28 LTO Roadmap

LTO Roadmap

http://ultrium.com/technology/roadmap.html

29 And data deduplication is the key to using more disk more cost

And data deduplication is the key to using more disk more cost

effectively!

30 IBM ProtecTIER Deduplication Family

IBM ProtecTIER Deduplication Family

Scalable Capacity and Performance

TS7650G & TS7680 ProtecTIER Gateways

TS7650 ProtecTIER Appliances

TS7620 ProtecTIER Appliance Express

Up to 2800 MB/sec Up to 1 PB Useable Capacity

Up to 500 MB/sec 7 TB to 36 TB Useable Capacity

Up to 145 MB/sec 5.5 TB and 11 TB Useable Capacity

Highest Performance Largest Capacity High Availability

Better Performance Larger Capacity Scalable

Good Performance Entry Level Easy to Install

31 Koninklijke Bibliotheek National Library of the Netherlands

Koninklijke Bibliotheek National Library of the Netherlands

During year 2000 IBM and KB projected and implemented a digital data preservation system called DIAS (Digital Information Archiving System). DIAS is the solution for the archiving and preservation of the multimedia and electronic digital-format documents. DIAS is compliant to the OAIS(1) standards related to the logical and physical preservation. IBM realized the DIAS solution using standard software components of general usage: WebSphere, DB2, Tivoli Storage Manager and Content Manager.

IBM DIAS - Digital Information Archiving System

(1) OAIS: http://public.ccsds.org/publications/archive/650x0b1.pdf Koninklijke Bibliotheek: http://www.kb.nl/dnp/e-depot/e-depot-en.html

32 Ecosystem: Thought Equity Motion Sports Video Archiving in the Cloud

Ecosystem: Thought Equity Motion Sports Video Archiving in the Cloud

Challenges Low cost delivery platform for enterprise scale Video Supply Chain as a Service Information growth of ~100 TB per month Easy self-serve access required by clients Solution IBM LTFS at several global locations, including some client facilities IBM System Storage TS3200 Tape Library, LTO-5 tape drives

LTO 5 and LTFS significantly reduce the ancillary costs around storage. This is a real game-changer from IBM Mark Lemmons CTO, Thought Equity Motion

Benefits Opened up new business opportunities Enabled more predictable and transparent pricing for clients Portable, interoperable, scalable, cost-effective data protection and long-term storage

TEM with LTFS on Youtube: http://www.youtube.com/watch?v=M7w0jrkQnj4

TSP03327-USEN-00

33 Thank you for your attention

Thank you for your attention

The IBM view on storage archive solutions: requirements to solve and trends for the future 31st
http://900igr.net/prezentacija/anglijskij-jazyk/the-ibm-view-on-storage-archive-solutions-requirements-to-solve-and-trends-for-the-future-31st-82657.html
c

25

29
900igr.net > > > The IBM view on storage archive solutions: requirements to solve and trends for the future 31st