Compact, Compress, De-DUplicate
•Download as PPT, PDF•
2 likes•1,201 views
Session held on 20-Sep-2010, AdminCamp 2010, Maritim Hotel, Gelsenkirchen Germany
![Über: Ulrich Krause ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Compact, Compress, De-DUplicate
Similar to Compact, Compress, De-DUplicate (20)
More from Ulrich Krause
More from Ulrich Krause (20)
Compact, Compress, De-DUplicate
- 1. Compact, Compress, De-Duplicate Ulrich Krause Admin Camp, 20. – 22.09.2010, Gelsenkirchen, Deutschland
- 16. Design Note Compression Design Text Attachments Design Text Attachments
- 19. Document Body Compression Design Text Attachments Design Text Attachments
- 23. LZ1 Attachment Compression Design Text Attachments Design Text Attachments
- 24. DAOS Estimator
- 28. D A O S
- 39. Das Endresultat Design Text Tickets Design Text Attachments DAOS Repository
Editor's Notes
- The reason for my concern? We have been told that the DAOS store has lesser disk performance demands than say the Transaction Logs or the data directory, yet it appears that when a DAOS MGR RESYNC or similar operation is initiated, actually a very intensive audit of the DAOS area takes place. This is where the 40,000 file per directory comes in... Just doing an 'ls' or 'dir' of a very large directory like that takes a significant time, particularly when this area is stored on NFS or iSCSI disk. In the case of a NFS mounted DAOS directory (say on an enterprise-class NetApps file), that 40000 files per subcontainer is too much. When the DAOS MGR RESYNC starts, it enumerates the contents of all the subcontainers. Due to the nature of NFS this is an extremely intensive operation. For 1.2 million attachments in 30 subcontainers this takes quite some time on NFS (3 hours or so) and is a heavy hit on the NFS filer performance. During this time, the daos catalogue daos cat.nsf is "unavailable", and so mail routing cannot occur until directory enumeration is complete. This is only going to get worse as more attachments are added to the DAOS store. Whenever the DAOS is resync'd, mail cannot be routed until this initial phase is completed and it starts on the databases themselves. In my opinion, we would see much better performance on initial resync if file count per subcontainer can be restricted to 5000 files or less for situations such as NFS. So if there isn't a means of decreasing the files/directory ratio now, can one be created? Feedback number SMIE82TFFY created by Stuart McIntyre on 19.02.2010
- SPR# VPRS7ZVTNM - If the DAOS storage path is moved under the regular Domino data hierarchy, it can cause DbDirMan to not detect that it is a DAOS subdirectory and so it will try to scan it while refreshing the cache of databases. When there are lots of files in DAOS, this can cause a performance drop while the scan is in progress. This fix will allow DbDirMan to identify and skip the DAOS data subdirectory when it has been moved into the Domino data hierarchy instead of away from it.
- Changing the DAOS minimum participation size does not have an effect on existing attachments...until you run a 'compact -c' operation. A 'compact -c' is essentially creating a new NSF, and copying every document and attachment from the old NSF. As each attachment is written to the new NSF, DAOS follows the current setting for the minimum participation size to decide if the attachment should be stored in DAOS or in the NSF. If you increase the minimum size, you get more attachments stored in the NSF, and fewer in DAOS, and decreasing the minimum size will have the opposite effect. In your case, since you increased the minimum participation size, the NSF will get bigger, and the NLO references will go down, but only after you run the 'compact -c' command. If any of the NLO files now have 0 references, they will be deleted by the prune operation when the deferred deletion interval has expired.
- How does DAOS functions with mail quotas? Solution DAOS will function with mail quotas depending on the quota enforcement setting that is in place. The quota enforcement settings available are: 1. Check space used in file when adding a note 2. Check filesize when extending the file 3. Check filesize when adding a note The caveat to these when using DAOS is that the attachments for the mail file are not in the mail file itself. Since this is the case, the attachments are not taken into account for space used, but are taken into account for the total file size. So if you are using "Check space used in file when adding a note", the quota will act on the physical size of the mail file (without attachments). Conversely if you are using either "Check filesize when extending the file" or "Check filesize when adding a note", quotas will act on the logical size of the mail file (taking into account attachments).
- http://www-01.ibm.com/support/docview.wss?rs=899&uid=swg21426481 New behavior in this release When the -daos on and -a options are used together, for example: load compact -a -DAOS on mailsmith.nsf and when an archive database is newly created during the archive, the new archive database will be DAOS-enabled. Note: The combination of compact options will not affect the mailsmith.nsf database itself, or any existing archive databases. Also, the behavior of compact does not change when the otions -a and -DAOS off are used together.