Data Coding

Multi-threaded erasure coding in Jerasure Cauchy Reed-Solomon

Background

Michael Jugan Tested with three different data layouts

*******@***.*** 2 KB PacketSize

One packet-column WPD packet-columns

Erasure Coding

D0

Goals

De ned: Applications:

a technique used in network data transmission

computer systems to le archiving Average speeds for all tested parameters (Hydra)

handle data loss bar code reliability

increase performance

and corruption 1000

Update Jearasure to utilize multiple processor cores while coding provide an intuitive user interface

Speed (MB/sec)

800

As the number of

Start with K blocks of data. 600

cores is increased,

1 core

it becomes vital to

400

Implementation

2 cores

use small packets.

3 cores

200

Encode to get M additional blocks of coding data. 4 cores

0

2KB PS WPD packet- One packet-

columns column

Four methods for splitting the work among threads: Packet Layout

Decoding recovers the data when up to M blocks are lost.

method name (max # of threads supported)

4 cores, 2KB PS

packets (PPS x M)

disks (M) packet_rows (WPD x M) packet_cols (PPS / WPD)

Cetus

Hydra

Jerasure - an open-source erasure coding library C0 1000 1000

800 800

Speed (MB/sec)

Reed-Solomon coding: C1 600 600

400 400

Data is organized in blocks. PacketSize (PS) Each color represents the data coded by a single thread 200 200

K = 2 PacketSize PacketSize D x PacketsPerSlice (PPS) 0 0

0

PPS = 2 PacketSize PacketSize D1 BlockSize 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

Two new public variables were added to class JER_Slices. Users add two additional lines of code before calling Encode . M

M

1 core, disks disks packet_rows packet_cols packets

- int NumberOfCores slices->NumberOfCores = 4;

W) - string MultiThreadMethod slices->MultiThreadMethod = disks ;

Performs matrix multiplication over Galois eld GF(2 Disks - fast when M is a multiple of NumberOfCores.

slices->Encode ;

Otherwise, performance su ers because some

D

1 0 0 Original Data Original Data 0

threads independently encode a second disk.

x =

K = 3 0 Original Data D1

Original Data

1 0

Packet_cols - consistently performs well when PS = 2KB

Original Data D2

0 Original Data

0 1

Performance tests and results

C

a b c 0

Coding Data

M = 2 C

d e f 1

Hydra Cetus

4 4

Testing Methods

Average Performance

Relative to One Core

3

Cauchy Reed-Solomon Coding: 3

Hydra Cetus

Encoded 100 MB with Reed-Solomon and Cauchy Reed-Solomon 2

Special case for W = 1 2

-O3 compiler optimizations

Galois eld arithmetic no longer needed Intel Xeon X5550 Intel Core 2 Quad Q9300

CPU

2.66 GHz Quad Core 2.55 GHz Quad Core 1 1

Averaged 5 runs where each run averaged 10 encodes

only memcpy and XOR

New data-layout parameter: words-per-drive (WPD) Bus speed 3200 MHz 1333 MHz

0 0

Coding Parameters: 2 cores 3 cores 4 cores 2 cores 3 cores 4 cores

L1 Cache 256 KB 256 KB

Number of cores Number of cores

Requirement: Reed-Solomon W = 8, 16, 32

K=1 Original Data L2 Cache 1 MB 6 MB

D Max speedup of 3.62x using 4 cores on Hydra

WPD WPD = 5, (the smallest valid value for

WPD = 2 Original Data 0 Cauchy Reed-Solomon

L3 Cache 8 MB -

2 +1 K+M Slower than the single-threaded case when large

all tested values of K and M

OS Ubuntu 10.04 64-bit Ubuntu 10.04 64-bit

packet-sizes are used

K = 10

Ram 12GB (1066 MHz DDR3) 4GB (800 MHz DDR2)

Both

Parallel programming M varied from 1 to 8

Memcpy speed 6.33 GB/s 2.77 GB/s

NumberOfCores increased from 1 to 4

Conclusion

XOR speed 4.53 GB/s 1.67 GB/s

In recent years, CPU speeds have leveled o due

Reed-Solomon

to device limitations such as power consumption.

Increased performance

Cetus

Hydra

Hydra, W = 8 Cetus, W = 8

Core 2 Extreme Core i5 4 4

700

700

3000

- 2 cores almost 2x speedup

Core 2 Quad

600

600

Frequency (MHz)

- depends heavily on the chosen coding parameters

Average Performance

Core 2 Duo

Relative to One Core

3 3

Speed (MB/sec)

500

500 1 core

2000

Easy to use

Pentium 4

400

400 2 cores 2

- requires two additional lines of code.

2

300

300 3 cores

1000

Pentium III

Future work

200 200 4 cores

Pentium II 1 1

Pentium

- automate the process of nding optimal settings

100 100

0

199*-****-**** 199*-****-**** 200*-****-**** 2010 2012 0 0

0 0

Sources: [0],[1] 2 cores 3 cores 4 cores

1 2 3 4 5 6 7 8 2 cores 3 cores 4 cores

1 2 3 4 5 6 7 8

M

M Number of cores Number of cores

References

Modern processor advancements

dual-core

Single-threaded encoding is extremely fast when M = 1. The performance for a set number of

focus heavily on multi-core CPUs.

- The rst row of the Reed-Solomon coding matrix contains all ones. cores varied depending upon the value of M.

Software must be specially programmed - Therefore, Galois multiplication is not used to encode the rst Altering W did not signi cantly change [0] "Intel Microprocessor Quick Reference Guide - Product Family,"

to take advantage of the multiple cores. http://www.intel.com/pressroom/kits/quickre am.htm.

coding drive; the data is simply copied and XOR d. the relative speedup.

[1] "Ark Your source for information on Intel products," http://ark.intel.com.

Contact this candidate