Java File I/O 在 Linux 上的效能比較 Part I - JCConf 2018

1. Java File I/O 大作戰Java File I/O 大作戰
JCConf 2018JCConf 2018
Michael Fong
1 . 1

2. MeMe
Software Engineer at
AP / Switch / IoT / System
Management and Controller / SaaS
If you are .
Software Development / Performance
Engineering
Ruckus Wireless - Part of Arris Group
interested
Github
Dockerhub
Slideshare
1 . 2

3. DisclaimerDisclaimer
Introductory level presentation
A bit material about Linux operating system
Mostly based on OpenJdk 1.8u
NOT going to cover performance measurement tools
Benchmark for reference ONLY. Source code available
on .
Correlated to settings from OS / hardware / cloud
service provider.
Recommended to write your own benchmark on
your test bed.
NOT an expert on many of the issues / areas.
github
1 . 3

4. java.iojava.io
2 . 1

5. Stream IOStream IO
Reader WriterSequence of (Unicoded) Character
InputStream OutputStreamStream of Bytes
2 . 2

6. Decorator PatternDecorator Pattern
Combination of BehaviorsCombination of Behaviors
[1] Decorator Pattern
2 . 3

7. Decorator PatternDecorator Pattern
Combination of BehaviorsCombination of Behaviors
Reader reader = new BufferedReader(new InputStreamReader(new FileInputStream("example.txt")));
int data = -1;
while ((data = reader.read()) != -1) {
//swallow
}
[1] Decorator Pattern
2 . 3

8. Decorator PatternDecorator Pattern
Combination of BehaviorsCombination of Behaviors
Reader reader = new BufferedReader(new InputStreamReader(new FileInputStream("example.txt")));
int data = -1;
while ((data = reader.read()) != -1) {
//swallow
}
read()
FileInputStream
raw byte data
[1] Decorator Pattern
2 . 3

9. read()
InputStreamReader
Decorator PatternDecorator Pattern
Combination of BehaviorsCombination of Behaviors
Reader reader = new BufferedReader(new InputStreamReader(new FileInputStream("example.txt")));
int data = -1;
while ((data = reader.read()) != -1) {
//swallow
}
read()
FileInputStream
raw byte data
unicoded byte data
[1] Decorator Pattern
2 . 3

10. read()
BufferedReader
read()
InputStreamReader
Decorator PatternDecorator Pattern
Combination of BehaviorsCombination of Behaviors
Reader reader = new BufferedReader(new InputStreamReader(new FileInputStream("example.txt")));
int data = -1;
while ((data = reader.read()) != -1) {
//swallow
}
read()
FileInputStream
raw byte data
unicoded byte data
buffered byte data
[1] Decorator Pattern
2 . 3

11. java.niojava.nio
2 . 4

12. Short History of NIOShort History of NIO
Introduced with JDK 1.4 to complement an existing
standard I/O.
Originally stands for Non-Blocking IO
Implementation
Support Channel / ByteBuffer
Support FileChannel / SocketChannel
Support Selector I/O Model
Continuous development was added to JDK 7 as
Including a series of new file system API as - aka
NIO.2
Implementation
Support Asynchronous I/O Model
2002
JSR-51
JSR-202
2 . 5

13. Java NIO In a Nut ShellJava NIO In a Nut Shell
Channel and Buffer
has more flexibility
Non Blocking Paradigm
*
Common in Network I/O
Selector Model (Network I/O)
ByteBuffer
AsynchrnousFileChannel
2 . 6

14. Buffer / CacheBuffer / Cache
3 . 1

15. Memory HierarchyMemory Hierarchy
CPU
Register
CPU Cache
L1 Cache
L2 Cache
Physical Memory
Random Access File
File Based Memory
Solid State Memory
Mechanical Hard Disk 3 . 2

16. Memory HierarchyMemory Hierarchy
CPU
Register
CPU Cache
L1 Cache
L2 Cache
Physical Memory
Random Access File
File Based Memory
Solid State Memory
Mechanical Hard Disk
0.5 ns
3 . 2

17. Memory HierarchyMemory Hierarchy
CPU
Register
CPU Cache
L1 Cache
L2 Cache
Physical Memory
Random Access File
File Based Memory
Solid State Memory
Mechanical Hard Disk
0.5 ns
7 ns
3 . 2

18. Memory HierarchyMemory Hierarchy
CPU
Register
CPU Cache
L1 Cache
L2 Cache
Physical Memory
Random Access File
File Based Memory
Solid State Memory
Mechanical Hard Disk
0.5 ns
7 ns
100 ns
3 . 2

19. Memory HierarchyMemory Hierarchy
CPU
Register
CPU Cache
L1 Cache
L2 Cache
Physical Memory
Random Access File
File Based Memory
Solid State Memory
Mechanical Hard Disk
0.5 ns
7 ns
100 ns
250 us - seq read 1mb
3 . 2

20. Memory HierarchyMemory Hierarchy
CPU
Register
CPU Cache
L1 Cache
L2 Cache
Physical Memory
Random Access File
File Based Memory
Solid State Memory
Mechanical Hard Disk
0.5 ns
7 ns
100 ns
250 us - seq read 1mb
3 . 2
1 ms - seq read 1 mb

21. Memory HierarchyMemory Hierarchy
CPU
Register
CPU Cache
L1 Cache
L2 Cache
Physical Memory
Random Access File
File Based Memory
Solid State Memory
Mechanical Hard Disk
0.5 ns
7 ns
100 ns
250 us - seq read 1mb
3 . 2
1 ms - seq read 1 mb
20 ms - seq read 1 mb

22. Memory HierarchyMemory Hierarchy
CPU
Register
CPU Cache
L1 Cache
L2 Cache
Physical Memory
Random Access File
File Based Memory
Solid State Memory
Mechanical Hard Disk
0.5 ns
7 ns
100 ns
250 us - seq read 1mb
3 . 2
1 ms - seq read 1 mb
20 ms - seq read 1 mb
Latency Comparison Numbers (~2012)

23. I/O Buffering EffectI/O Buffering Effect
3 . 3

24. I/O Buffering EffectI/O Buffering Effect
Main
Memory
Memory Buffer
Disk
3 . 3

25. I/O Buffering EffectI/O Buffering Effect
Main
Memory
Memory Buffer
Disk
3 . 3
Cache Miss
Populate Page

26. I/O Buffering EffectI/O Buffering Effect
Main
Memory
Memory Buffer
Disk
3 . 3
Cache Miss Cache Hit
Populate Page

27. I/O Buffering EffectI/O Buffering Effect
Main
Memory
Memory Buffer
Disk
3 . 3
Cache Miss
Speed up READ performance
Cache Hit
Populate Page

28. I/O Buffering EffectI/O Buffering Effect
Main
Memory
Memory Buffer
Disk
3 . 3
Cache Miss
Speed up READ performance
Cache Hit
Populate Page

29. I/O Buffering EffectI/O Buffering Effect
Main
Memory
Memory Buffer
Disk
3 . 3
Cache Miss
Speed up READ performance
Cache Hit
Populate Page

30. I/O Buffering EffectI/O Buffering Effect
Main
Memory
Memory Buffer
Disk
3 . 3
Cache Miss
Speed up READ performance
Speed up WRITE performance
Cache Hit
Populate Page

31. I/O Data PathI/O Data Path
I/O Data path is consisted of layers of buffers for
read and write.
Systems performance is the study of the entire
system, including all physical com-
ponents and the full software stack. [1]
pages from disk to memory -
()
Not in Cache: ()
In Cache: and exit
pages to disk - ()
FileSystem specific implementation
Read
do_generic_file_read
page_cache_sync_readahead
Copy page
Write generic_perform_write
[1]Systems Performance - Enterprise and the Cloud ( 2014)
3 . 4

32. stdstd
read()read()
3 . 5

33. stdstd
write()write()
Write Back /
Delayed Write
3 . 6

34. mmapmmap
Read: An occurred page fault
will bring in the file data from
disk.
The affected page is marked
as dirty and will be eventually
flushed to file on disk.
Pros:
Avoid extra data copy
Minimize kernel/user
context switch
Cons:
Page faults may bring
to
performance
negative impact
3 . 7

35. sendfile()sendfile()
3 . 8

36. sendfile()sendfile()
First appeared in kernel v2.2
file data to be copied into kernel
buffer directly by the DMA engine.
3 . 8

37. sendfile()sendfile()
For kernel v2.4+ supporting
Scatter/Gather operations
file descriptor and meta data are
passed to socket buffer.
The DMA engine passes data
directly from the kernel buffer to
the protocol engine,
First appeared in kernel v2.2
file data to be copied into kernel
buffer directly by the DMA engine.
3 . 8

38. sendfile()sendfile()
For kernel v2.4+ supporting
Scatter/Gather operations
file descriptor and meta data are
passed to socket buffer.
The DMA engine passes data
directly from the kernel buffer to
the protocol engine,
First appeared in kernel v2.2
file data to be copied into kernel
buffer directly by the DMA engine.
Zero Copy
3 . 8

39. Deep Dive OpenJDKDeep Dive OpenJDK
(8u) Source Code(8u) Source Code
4 . 1

40. FileInputStreamFileInputStream
Read Path - jdk8uRead Path - jdk8u
1.
2.
3.
4.
5.
FileInputStream.java
FileInputStream.c
io_util.c
io_util_md.c
io_util_md.h
4 . 2

41. FileOutputStreamFileOutputStream
Write Path - jdk8uWrite Path - jdk8u
1.
2.
3.
4.
5.
FileOutputStream.java
FileOutputStream_md.c
io_util.c
io_util_md.c
io_util_md.h
4 . 3

42. FileChannel Memory Map Path - jdk8uFileChannel Memory Map Path - jdk8u
FileChannel.MapMode Read File Write File C Permission C Flags
READ_ONLY Yes ReadOnlyBufferExce
ption
PROT_READ MAP_SHARED
READ_WRITE Yes Yes PROT_WRITE | PROT_READ MAP_SHARED
PRIVATE Yes Yes (in memory only) PROT_WRITE | PROT_READ MAP_PRIVATE
1.
2.
FileChannelImpl.java
FileChannelImpl.c
4 . 4

43. FileChannel TransferTo Path - jdk8uFileChannel TransferTo Path - jdk8u
4 . 5

44. Data PersistenceData Persistence
Page CachePage Cache
As of Linux kernel 2.4, there is only one Page Cache
Write Path
Data is firstly written in page cache, called Dirty Pages:
When free memory shrinks below threshold - Flusher Threads
When dirty data grows older than specific threshold - Flusher
Threads
When user invokes sync() / fsync() on demand
Read Path
a read() would check page cache in memory before accessing
the disk.
[1]
4 . 6

45. Data PersistenceData Persistence
FileChannel.force() native syscall
metaData=True fsync()
metaData=False fdatasync() 4 . 7

46. Data PersistenceData Persistence
RandomAccess
File()
native syscall
mode="rws" flags|=O_SYNC
mode="rwd" flags|=O_DSYNC
4 . 8

47. Benchmark SetupBenchmark Setup
5 . 1

48. Setup 1Setup 1
Host
Hypervised Jenkins
Intel Skylake 3.5GHz 2-core CPU
8 GB Ram
20 GB Solid State Device
Intel i7 X-Series CPU
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
TEST 128G 232028 33 142119 26 448889 47 692.2 71
Latency 4051ms 596ms 1400ms 44991us
Version 1.97 ------Sequential Create------ --------Random Create--------
TEST -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 235 5 +++++ +++ 238 3 237 5 +++++ +++ 236 3
Latency 188ms 238us 21373us 29217us 41us 48428us
5 . 2

49. Setup 2Setup 2
Host
Seagate Barracuda 500GB w/ software RAID-10
KVM Hypervised Jenkins
Intel 2.7GHz 2-Core CPU
8 GB Ram
80 GB SATA (7200 RPM)
Intel Core i5-6400
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
Bonnie++Benchma 16G 26705 3 2350 0 21671 1 94.5 4
Latency 215ms 34089ms 809ms 776ms
Version 1.97 ------Sequential Create------ --------Random Create--------
Bonnie++Benchmark -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 36 0 +++++ +++ 37 0 37 0 +++++ +++ 35 0
Latency 865ms 439us 735ms 774ms 191us 1329ms
5 . 3

50. Java Microbenchmark HarnessJava Microbenchmark Harness
is a Java harness for building, running, and
analysing nano/micro/milli/macro benchmarks
written in Java and other languages targetting the
JVM.
Common Microbenchmark Pitfall
Manage JVM optimizations
Warmup Stage
Dead Code Elimination
supported
JMH
Jenkins style report
5 . 4

51. SequentialSequential
Read / WriteRead / Write
BenchmarkBenchmark
6 . 1

52. Overall BenchmarkOverall Benchmark
java.io
BufferedStream
FileStream w/ byte[]
w/ byte[]RandomAccessFile
java.nio
FileChannel#
FileChannel#
FileChannel#
FileChannel
AsynchnousFileChannel
map
transferTo
transferFrom
NUM_OF_WARM_UP = 10
6 . 2

53. Benchmark SummaryBenchmark Summary
java.io Experiment user buffer
copyWithBufferedFileStream
copyWithRawBuffer byte[]
copyWithRawBufferedRand
omAccessFile
byte[]
java.nio Experiment user buffer mmap() sendfile()
copyWithMmap Y
copyWithAsyncFileChannel ByteBuffer
zeroTransferToCopy Y (conditional) Y (conditional)
zeroTransferFromCopy Y
copyWithFileChannel ByteBuffer
6 . 3

54. Each ExperimentEach Experiment
# Drop page cache, dentry cache and inode
$ echo 3 > /proc/sys/vm/drop_caches
buffer size = { 32 , 256, 1024, 8192 , 102400} bytes
File size = 32mb
NUM_OF_ITERATION= 20
6 . 4

55. Benchmark ResultBenchmark Result
Source Code 6 . 5

56. Random AccessRandom Access
Read / WriteRead / Write
BenchmarkBenchmark
7 . 1

57. Overall BenchmarkOverall Benchmark
java.io
FileStream w/ byte[]
java.nio
FileChannel#map
7 . 2

58. Each ExperimentEach Experiment
# Drop page cache, dentry cache and inode
$ echo 3 > /proc/sys/vm/drop_caches
NUM_OF_ITERATION = 20
input.data
input.idx
{offset}{len}{type}
0, 1024, string
....
33443, 245, int
benchmark.jar
name.output
numbers.output
....output
7 . 3

59. Benchmark ResultBenchmark Result
Source Code 7 . 4

60. File ReplicationFile Replication
BenchmarkBenchmark
8 . 1

61. Overall BenchmarkOverall Benchmark
NUM_OF_WARM_UP = 10
java.nio.file.Files#
java.nio.channels.FileChannel#
com.google.common.io.Files#
org.apache.commons.io.FileUtils#
copy
transferTo
copy
copyFile 8 . 2

62. Each ExperimentEach Experiment
# Drop page cache, dentry cache and inode
$ echo 3 > /proc/sys/vm/drop_caches
NUM_OF_ITERATION = 20
file size = { 1 , 10, 100 } mega bytes
8 . 3

63. Benchmark ResultBenchmark Result
Source Code8 . 4

64. How ZeroCopy OutperformsHow ZeroCopy Outperforms
8 . 5

65. How ZeroCopy OutperformsHow ZeroCopy Outperforms
CPU %
Disk Write
Op / sec
8 . 5

66. How ZeroCopy OutperformsHow ZeroCopy Outperforms
CPU %
Disk Write
Op / sec
8 . 5

67. How ZeroCopy OutperformsHow ZeroCopy Outperforms
Files.copy()
CPU %
Disk Write
Op / sec
8 . 5

68. How ZeroCopy OutperformsHow ZeroCopy Outperforms
Files.copy() FileChannel.transferTo()
CPU %
Disk Write
Op / sec
Data
Preparation
8 . 5

69. Some Use CaseSome Use Case
9 . 1

70. Apache KafkaApache Kafka
Consumer Producer
Topic
Each partition is a
Every message is appended at the end of each partition.
Thus, each consumer remembers an offset of which
partition it has recently consumed for each topic.
Each message is transmitted via to transmit the
partition file with offset + length.
memory mapped file
sendfile()
9 . 2

71. ApacheApache CassandraCassandra
C* is known to provide high write throughput not only because
of its overall architecture, but also it fundamentally utilize
mmap() to appends data to , or from the
immutable SSTable file on disk
Commit Log read data
9 . 3

72. C* Memory-Mapped FileC* Memory-Mapped File
root$ pmap -x <pid> | grep "CommitLog"
00007ff8d4423000 32768 32768 0 rw-s- CommitLog-3-1537419962547.log
00007ff8d6423000 32768 32760 0 rw-s- CommitLog-3-1537419962545.log
00007ff8e798c000 32768 32768 0 rw-s- CommitLog-3-1537419962567.log
00007ff8f598d000 32768 24088 24 rw-s- CommitLog-3-1537419962575.log
00007ff8f798d000 32768 32768 0 rw-s- CommitLog-3-1537419962574.log
00007ff8f998d000 32768 4 0 rw-s- CommitLog-3-1537419962576.log
00007ff8fb98d000 32768 32768 0 rw-s- CommitLog-3-1537419962573.log
00007ff8fd98d000 32768 32768 0 rw-s- CommitLog-3-1537419962562.log
00007ff8ff98d000 32768 32768 0 rw-s- CommitLog-3-1537419962561.log
00007ff902294000 32768 32768 0 rw-s- CommitLog-3-1537419962566.log
00007ff904294000 32768 32768 0 rw-s- CommitLog-3-1537419962565.log
00007ff90798d000 32768 32768 0 rw-s- CommitLog-3-1537419962564.log
-bash-4.1$ sudo pmap -x <pid> | grep -E ".db"
00007ff930d5c000 4 0 0 r--s- system-peers-jb-30-Index.db
00007ff934898000 4 0 0 r--s- system-schema_columns-jb-365-Index.db
00007ff935985000 4 0 0 r--s- system-schema_keyspaces-jb-366-Index.db
00007ff9382b1000 4 0 0 r--s- system-local-jb-74-Index.db
00007ff9382b2000 4 4 0 r--s- system-local-jb-73-Index.db
00007ff9382ba000 4 0 0 r--s- system-schema_triggers-jb-5-Index.db
00007ff944f6e000 80 0 0 r--s- system-compaction_history-jb-882-Index.
00007ff944f82000 32 0 0 r--s- system-sstable_activity-jb-1058-Index.d
00007ff9482f5000 32 0 0 r--s- system-sstable_activity-jb-1059-Index.d
00007ff94830d000 4 0 0 r--s- system-compactions_in_progress-jb-18350
00007ff94838e000 4 0 0 r--s- system-compactions_in_progress-jb-18352
00007ff94838f000 4 0 0 r--s- system-compactions_in_progress-jb-18351
00007ff94a19f000 4 0 0 r--s- system-peers-jb-31-Index.db
00007ff94a5af000 4 0 0 r--s- system-peer_events-jb-105-Index.db
Read+Write+Shared
Read+Shared
9 . 4

73. 10 . 1

74. ReferenceReference
Source Code Repository
Books
Advanced Programming in the UNIX Environment -
3rd Edition (2013)
Linux System Programming - 2nd Edition (2013)
Linux Kernel Development - 3rd Edition (2011)
Java Performance - The Definitive Guide (2014)
Systems Performance - Enterprise and the Cloud (
2014)
OpenJdk 8u
Linux Source v2.6.39
10 . 2

75. BackupBackup
11 . 1

76. IO Buffer - Car PoolingIO Buffer - Car Pooling
Traffic = NumberofPassenger/Second
IOThroughput = BytesTransferred/Se
[1] Image Source
11 . 2

77. SwapSwap
The kswapd daemon (v2.x+) is also known as the pageout daemon. It
supports the virtual memory subsystem by writing dirty pages to disk
slowly over time, so the pages can be reclaimed.
Disabling Swap Space
Suggested for many server application, i.e. ,
does not prevent disk I/O from becoming a problem under memory
contention, it simply shifts the disk I/O thrashing from
(MAP_PRIVATE) to (MAP_SHARED).
does not prevent pathological behaviour at near-OOM, although
having swap space may prolong it. Whether the system OOM killer
is invoked with or without swap, the result is the same: you are left
with a system in an unpredictable state.
ElasticSearch Cassandra
anonymous
pages file pages
11 . 3

78. Direct IODirect IO
Currently NOT supported, but possibly
available in
JNA/JNI compliant
Pros
IO throughput is more predictable:
bypass the influence of page cache.
Cons
Slow down reads, since bypassing
page cache completely.
Check
A lot of custom solution
JDK 10
C* ticket
Ignite-direct-io
jaydio
Write Through
11 . 4

79. PageCachePageCache
Page Writeback SettingsPage Writeback Settings
dirty_background_ratio As a percentage of total memory,
the number of pages at which
the flusher threads begin
writeback of dirty data.
dirty_expire_interval In milliseconds, how old data must
be to be written out the next time a
flusher thread wakes to perform
periodic writeback.
dirty_ratio As a percentage of total memory,
the number of pages a process
generates before it begins
writeback of dirty data.
dirty_writeback_interval In milliseconds, how often a flusher
thread should wake up to write
data back out to disk. 11 . 5

80. Performance MeasurementPerformance Measurement
Latency vs Throughput vs BandwidthLatency vs Throughput vs Bandwidth
Time (sec)
Event
Throughput is how much event per unit time travelled through the channel
Latency is how much time for an event to
complete
Bandwidth is maximum of event
traveled through the channel
11 . 6

81. Application BufferApplication Buffer
BufferedOutputStreamBufferedOutputStream
public class BufferedOutputStream extends FilterOutputStream {
/** Flush the internal buffer */
private void flushBuffer() throws IOException {
if (count > 0) {
out.write(buf, 0, count);
count = 0;
}
}
public synchronized void flush() throws IOException {
flushBuffer();
out.flush();
}
}
11 . 7

82. Files CopyFiles Copy
Path - jdk8uPath - jdk8u
1.
2.
3.
Files.java
UnixCopyFile.java
UnixCopyFile.c
11 . 8

83. Java APIJava API
Data PersistenceData Persistence
11 . 9

84. copyWithRawBuffercopyWithRawBuffer
public void copyWithRawBuffer(SequentialReplicationExecutionPlan plan) throws IOException {
try(
FileInputStream fin = new FileInputStream(plan.finPath);
FileOutputStream fout = new FileOutputStream(plan.foutPath);
) {
byte[] buffer = new byte[plan.bufferSize];
int numBytesRead = 0;
while ((numBytesRead = fin.read(buffer)) != -1) {
fout.write(buffer, 0, numBytesRead);
}
}
}
11 . 10

85. copyWithBufferedFileStreamcopyWithBufferedFileStream
public void copyWithBufferedFileStream(SequentialReplicationExecutionPlan plan) throws IOExc
try(
BufferedInputStream fin = new BufferedInputStream(new FileInputStream(plan.finPath),
BufferedOutputStream fout = new BufferedOutputStream(new FileOutputStream(plan.foutP
) {
int byteRead = 0;
while ((byteRead = fin.read()) != -1) {
fout.write(byteRead);
}
fout.flush();
}
}
11 . 11

86. copyWithFileChannelcopyWithFileChannel
public void copyWithFileChannel(SequentialReplicationExecutionPlan plan) throws IOException
try(
FileChannel finChannel = new FileInputStream(plan.finPath).getChannel();
FileChannel foutChannel = new FileOutputStream(plan.foutPath).getChannel();
) {
int finLength = (int) finChannel.size();
for (int bufIndex = 0; bufIndex < finLength; ) {
ByteBuffer buffer = ByteBuffer.allocate(plan.bufferSize);
int bufLength = 0;
if (bufIndex + plan.bufferSize > finLength) {
bufLength = finLength % plan.bufferSize;
} else {
bufLength = plan.bufferSize;
}
finChannel.read(buffer);
//switch to write mode for ByteBuffer
buffer.flip();
foutChannel.write(buffer);
bufIndex += bufLength;
}
}
}
11 . 12

87. copyWithMmapcopyWithMmap
public void copyWithMmap(SequentialReplicationExecutionPlan plan) throws IOException {
try (
RandomAccessFile fin = new RandomAccessFile(plan.finPath, "r");
RandomAccessFile fout = new RandomAccessFile(plan.foutPath, "rw");
FileChannel finChannel = fin.getChannel();
FileChannel foutChannel = fout.getChannel();
) {
int finLength = (int) finChannel.size();
MappedByteBuffer bufIn = finChannel.map(FileChannel.MapMode.READ_ONLY, 0, finLength)
MappedByteBuffer bufOut = foutChannel.map(FileChannel.MapMode.READ_WRITE, 0, finLeng
for (int bufIndex = 0; bufIndex < finLength; ) {
int bufLength = 0;
if (bufIndex + plan.bufferSize > finLength) {
bufLength = finLength % plan.bufferSize;
} else {
bufLength = plan.bufferSize;
}
byte buffer[] = new byte[bufLength];
bufIn.get(buffer, 0, bufLength);
bufOut.put(buffer);
bufIndex += plan.bufferSize;
}
11 . 13

88. copyWithAsyncFileChannelcopyWithAsyncFileChannel
public void copyWithAsyncFileChannel(SequentialReplicationExecutionPlan plan) throws Excepti
try(
AsynchronousFileChannel finChannel = AsynchronousFileChannel.open(Paths.get(plan
AsynchronousFileChannel foutChannel = AsynchronousFileChannel.open(Paths.get(pla
) {
int finLength = (int) finChannel.size();
BlockingQueue<Boolean> isBufferComplete = new ArrayBlockingQueue<>(1);
for (int bufIndex = 0; bufIndex < finLength; ) {
ByteBuffer buffer = ByteBuffer.allocate(plan.bufferSize);
int bufLength = 0;
if (bufIndex + plan.bufferSize > finLength) {
bufLength = finLength % plan.bufferSize;
} else {
bufLength = plan.bufferSize;
}
finChannel.read(buffer, 0).get();
final int position = bufIndex;
//switch to write mode for ByteBuffer
buffer.flip();
foutChannel.write(buffer, bufIndex, isBufferComplete, new CompletionHandler<Inte
@Override
public void completed(Integer result, BlockingQueue<Boolean> lock) {
LOG.debug("streamed [{}] / [{}] bytes w/ buffer size [{}]", new Object[]11 . 14