Think sync (July 20, 2003) 同期を考えよう

Think the Synchronization!

ari (masanari Motohasi)
Institute of Mobile PC Technology

Purpose
●

How many PC or PDA do you have?
–

Office, Home, Mobiles ... any environments...

–

We wish that I had access one Information anywhere.
●

–

the complete networking

However, there are the barriers by cost, security, out of
range...

●

Do you have backup routinely?

●

We need to synchronize the information!

replication system
●

high availability ( 高可用性 )
–

●

redundancy ( 冗長化 ) / ritualized

consistency / concurrency ( 一貫性 )
–

contention ( 衝突回避 )

●

response time / reasonable ( 高性能 )

●

easy to use / manage

●

warm or cold standby

ultimately ubiquitous network
●

インターネットにどんなところからでもアクセス
–
–
–

●

いつでも、どこでも、どんな方法でも
分散されていて、もっとも正規化が進む
情報の同期は考えなくていいのでは？

しかし、ネットワーク的につながっている必要がある
–
–
–

セキュリティとプライバシは大丈夫？
同じセグメントかファイアウォールを抜く必要
コスト、圏外、公衆電話？

Viewpoints
●

like network layers (vertical)
–

●

physical/hardware to applications/software

Transparency or not (horizontal).
–

normalized and distributed

–

“use caches” have the contended problem.

–

“make copies (backups)” have a lag time.

–

recovery time is very important
●
●

clear the “contention problem”.
hot or cold standby?

Application/Software

Directions
dump/load
database

web [page/service]
grid/(mobile) agent
telnet
[process/object] remote display
groupware / mail
[communication]
Version Controls[file]

file sync. (rsync/ftpsync)

backup [file]
(Tape etc)
[unit]

Database
[transaction]
Transparency

remote access
inet (VPN / SSH)
P2P
dial up [any]
mobile PC/PDA [any]
SAN / NAS /
SAN
Removable Media Internet Disk
(snapshot)
HDD/CF/SD..[file]
RAID 1|5
Physical/Hardware

Web pages and services
●

●

one of the most distributed and normalized
solution with the Internet
standard in the world
–
–

●

“standard” HTML / XML / SOAP etc
very reasonable

with secure
–

SSL (server|client certification, encryption)

–

and the other

–

only good design (^_^;

Web pages – Just Try It!
●

say nothing, because everybody knows (^ ^;)

Web Services – Just Try It!
●

SOAP / XML on HTTP
–

Apache SOAP (Java), Microsoft SOAP toolkit, SOAP
module(Perl)

–

connect to the other world.
●

●
●

Internet Applications (100% pure Java) and legacy systems
(Microsoft COM (ASP / VB / C)).
“Alive Check” system for an services (SOAP).
B2B applications (XML on HTTP POST)
SOAP

Microsoft COM

J2EE

SOAP
(HTTP+VPN)

B2B
another sites

Remote Console, Remote Display
●

Remote Console
–

●

telnet (mainly UNIX)

Remote Display / Desktop
–

VNC (Virtual Network Computing) *
●

–

pcAnywhere *
●

–

Windows only

X
●

–

Windows and Unix

mainly UNIX

Terminal Server (Windows)

Remote Console, Just Try It
●

telnet and telnetd (Unix)

●

Services for Unix (Windows) 29,800yen
–

functions
●
●
●

–

telnet daemon (password sync Windows to Unix)
server/client/gateway for NFS
server for NIS

for developers
●

all developers telnet, compile and test the applications

Remote Display, Just Try It
●

VNC (Virtual Network Computing)
–
–

●

for developers and tests only
check the security

pcAnywhere
–

control and administer to the servers on the branches
●

●

for security patch and maintenance

terminal service (Windows 2000)
–

another profiles

Groupware / Mail
●

communication!
–

●

from mails to workflow systems

Internet Mail System (on SMTP/POP3/IMAP etc)
–
–

●

very simple protocol
MIME

Exchange / Notes
–

has replication options

–

share files, meeting agents, but depend on the special
platform

Internet Mail, Just Try It!
●

Everybody knows in many OS and devices
–
–

●

Windows, Macintosh, UNIX, PDA, Phones, JRE...
devices and nodes

for the synchronized files
–

send the attached mail oneself. (MIME)

–

it's not useful (not seamless and automatically)
●

–

see “mailtrek” ...

however the firewall problem is clear! (with encrypt)
●

check the contents and words

Groupware, Just Try It
●

Microsoft Exchange
–

●

only Microsoft world( ?NET technology)

Louts Notes / Domino
–
–

●

closed technology and culture (not open).
the Intranet and workflow systems

have rich functions
–

●

mail system, scheduler, address books, public files,
databases, replications

depend on one company ... risk!

Database Management System
●

to be one of most normalized
–

●

keep isolation by “transaction”

SQL is the standard in the world
–

Oracle, Sybase ASE, Postage SQL, MS SQL...

●

but mainly use in the LAN.

●

2 phase commit

●

replication server
–

transaction base.

Database, Just Try It
●

Sybase Adaptive Server Enterprise 11.0 – 12.5 *

●

Oracle8i *

●

Microsoft SQL Server *

●

My SQL

●

PostgreSQL

replication – mirroring
●

write the many devices as same time
applications / users
read/write
controller

controller
WAN?

databases / devices / repositories(?)

replication – copy/ snapshot
●

write the transaction to the primary
–

copy / snapshot to the secondary (only one way)

read / write
copy
dump
snapshot

replication - data trigger
●

the trigger
–

one way

–

performance overhead

–

very complexly

–

rollback is not easy

read/write

! data updated
write

replications – stored & forward

difference
dump transaction

queue, DB, file, transaction log

2 phase commit
●

2 phase commit protocol is not redundant
–

just distributed

are you ready?

yes

yes

NO

replications – bi-directions
site A

site B

site A

site B

read/write

read/write

read/write

read/write

!
contention
read only

read only

make the contentions
●

when two or more “persons” update one
information as same time.
–

objects(Beans), file, transaction, page, law...
time

A
B

?

update

open

open

update!

contention – lock
●

blocking oriented
–

DBM, File Systems

–

use the wait queue

–

deadlock
lock

B (lock waits)

update

open
update

wait

open

A

time

B

contention Optimistic Concurrency Control
●

restart oriented
–

CVS, H/A EJB

time
update

re-open

B

×

update

open

open

A

B

Version Controls
●

not only “source” but “documents”

●

CVS [Concurrent Versions System]
–
–

shell (command prompt) and GUI

–

over HTTP, FTP, MAIL.

–

ported to any OS, but mainly in UNIX

–
●

optimistic concurrency control method

controlling my documents.

Microsoft Visual SourceSafe
–

for developers

File sync / replications
●

rsync

●

Microsoft Products
–

Briefcase (Windows 95 -) *

–

Replication Service (Windows NT)

–

Offline File Synchronization (Windows 2000 -)

–

Microsoft ActiveSync *
●

●

Desktop PC – Pocket PC (serial / infrared)

another products

rsync, Just Try It!
●

rsync on NT (run as services)
–

●

easy?

rsync algorithm
–

low-bandwidth high-latency bi-directional
communications

–

great

–

maybe not secure, not use in the naked Internet...

Microsoft Briefcase, Just Try It
●

Briefcase
–

–

ブリーフケースは直接ケーブル接続またはリムーバブルディ
スクを使用してコンピュータ間のファイル転送を頻繁に行う
場合に最適なツールです。ブリーフケースを使用して、メイ
ンコンピュータ上のファイルと他のコンピュータ上で変更した
ファイルを同期させることができます。複数のブリーフケース
を作成してファイルを整理することができます
easy to use, but performance is no good (many files
with Gateway2000 Handbook).

Microsoft Offline Files
●

●

オフラインファイルはネットワーク上の共有ファイルを操
作するのに最適なツールです。オフラインファイルを
使用すると、ネットワークに接続していないときに変
更した共有ファイルを次にネットワークに接続したとき
に同期させることができます。
使ったことがある方、どんな感じでしょう？

Microsoft Directory Replication
Service
●

“Windows NT Core Technology” で知りましたが、
実際の運用で使用したことはありません。
–

使ったことがある方、どんな感じでしょう？

Microsoft ActiveSync, Just Try It
–

Desktop PC – Pocket PC (serial / infrared)

–

Outlook data (mails, schedules, tasks, addresses...) and
files

–

easy to use
●

but I am used to erased all data by miss operations.

ftptrek / mailtrek, Just Try It!
●

ftptrek (Java)
–

synchronize with the data
●
●
●

faster than Windows(SMB) copy
check the size and hash of the file
support the zipped, parallel and serial copies

–
–
●

synchronize with database transaction
distribute over the program file updated

mailtrek (Java)
–

SOAP on SMTP/POP3/IMAP? HTTP on MIME?

–

Sorry, under construction

Remote Access
●

very basic technology (PPP, SLIP...)
–
–

●

dial-up to the node (ex. office or home)
via Internet access

the any services on PPP
–

wired and seamless!

Remote Access (Internet)
●

same as dial-up, but NOT secure

●

SSH / VPN
–

need: the secure server on the Internet

–

need: Global IP, Static IP or Dynamic DNS

–

need: keep the security and the privacy
●
●

–

ex. Many site that has old version OpenSSH is attacked.
ex. Do dial-up users need to open the port 23(telnet) for the
Internet?

SoftEther Virtual Ethernet System
●

ex. Ethernet on the HTTPS and SSH

VPN, Just Do It
●

IPsec (commercial)
–
–

●

CISCO VPN 3005
connected to another site

IPsec (private)
–

under construction

–

with FTTH (B-flets, NTT)

–

share the files, administrate the server

SoftEther Vitual Ethernet System
●

SoftEther Virtual Ethernet System
●
●
●

Ethernet on the HTTPS and SSH
Able to pass through the firewall
Take care for use.

mobile PC / PDA
●
●
●

皆さん、いろいろな使い方を教えてください！
notebook PC
PDA

Storages
●

SAN (Storage Area Networks)
–

fast/virtual Access (Fiber Channel)

–

snapshot make the backup in a second

●

Network Attached Storages

●

Internet Disk
–

use the Virtual Disk on the Internet Servers

SAN, Just Try It
●

databases and file servers
–

●

critical missions

snapshots is great
–

fast access (fiber channel and huge cache)

–

the backups and recoveries are in a second.

–

but the recoveries need the time
●

full backup a day, and “loading transactions” are needed.

NAS (at home), try it
●

HD-HLAN, Buffalo (Link Station)
–

change from the file server to HLAN (NAS)

–

low power, small size and silent

–

schedule backups the file a day via “rsync”

Godzilla
file server
RAID 5
20GB
(4G×6)

HLAN
no redundancy
160GB

Internet Hard Disk, Just Try It
●

accessible to the disk like a drive

●

test the “MYDISK” service of rimnet.
–

like the disk drive on the Windows
●
●
●

–

no accessible through our firewall
●

–

web accessible
100MB, free
show demo...
need the authentication to proxy

stable?

Removable Devices
●

HDD, Flash Memory(CF/SD)...
–

use as primary devices and backups

–

has the risks
●
●

●

if lost (stop the job?)
if stolen (get out the information?)

encryption
–

Encrypting File System (EFS) on windows 2000
●

–

over NTFS 5.0

PGPdisk

RAID 1 or 5
●

HDD is one of the most fragile parts
–

redundancy the HDD

Backup, Just Do It
●

Tape
–
–

serial access

–
●

Just standard, but very expensive for personal user
DLT / DLT / LTO...

HDD
–

Random access

–

I use the bus-powered 2.5inch HDD with rsync (at
home)

Enterprise Disaster Recover Site,
Just Do It
●

for Business Continuity Planning (BCP)
–

●

Reduce the risk! plug off?

redundant the anything
–
–

Wires, networks (include Internet) and servers

–
●

Database synchronized by ftptrek
humans and business processes

warm standby
–

if “the incident” is happened, switch in a minute.

Think sync (July 20, 2003) 同期を考えよう

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Think sync (July 20, 2003) 同期を考えよう

Similar to Think sync (July 20, 2003) 同期を考えよう (20)

More from Masanari Motohasi

More from Masanari Motohasi (13)

Recently uploaded

Recently uploaded (20)

Think sync (July 20, 2003) 同期を考えよう