From gedetil at cs.umanitoba.ca Tue Oct 6 11:02:30 2009 From: gedetil at cs.umanitoba.ca (Gilbert E. Detillieux) Date: Tue, 6 Oct 2009 11:02:30 -0500 (CDT) Subject: [RndTbl] MUUG Meeting, Oct 13, 7:30pm -- Jabber Message-ID: <200910061602.n96G2UJ00140@iron.cs.umanitoba.ca> The Manitoba UNIX User Group (MUUG) will be holding its next monthly meeting on Tuesday, October 13. The meeting topic for this month is as follows: Jabber Today, instant messaging is taken for granted. We use it to stay in constant contact with our friends and colleagues. There are some environments where it is not feasible or practical to use external IM services in a corporate environment. Like most things, there is an Open Source solution for that. In this case, we're talking about the Jabber instant messaging protocol. In this presentation, Darren Hildebrand and Sean Cody, from Prime Focus VFX, will talk about the Jabber system and show how the eJabberd implementation can be used to provide closed instant messaging services in a secure, simple and client-application agnostic way. They will also discuss what logging, auditing and authentication options are available. If you were ever interested in replacing or providing AIM/MSN/ICQ/gTalk functionality in your workspace this presentation is for you. The group holds its general meetings at 7:30pm on the second Tuesday of every month from September to June. (There are no meetings in July and August.) Meetings are open to the general public; you don't have to be a MUUG member to attend. ********************************************************************** Please note our meeting location: The IBM offices, at 400 Ellice Ave. (between Edmonton and Kennedy). When you arrive, you will have to sign in at the reception desk, and then wait for someone to take you (in groups) to the meeting room. Please try to arrive by about 7:15pm, so the meeting can start promptly at 7:30pm. Don't be late, or you may not get in. (But don't come too early either, since security may not be there to let you in before 7:15 or so.) Non-members may be required to show photo ID at the security desk. Limited parking is available for free on the street, either on Ellice Ave. or on some of the intersecting streets. Indoor parking is also available nearby, at Portage Place, for $5.00 for the evening. Bicycle parking is available in a bike rack under video surveillance located behind the building on Webb Place. ********************************************************************** For more information about MUUG, and its monthly meetings, check out their Web server: http://www.muug.mb.ca/ Help us promote this month's meeting, by putting this poster up on your workplace bulletin board or other suitable public message board: http://www.muug.mb.ca/meetings/MUUGmeeting.pdf -- Gilbert E. Detillieux E-mail: Manitoba UNIX User Group Web: http://www.muug.mb.ca/ PO Box 130 St-Boniface Phone: (204)474-8161 Winnipeg MB CANADA R2H 3B4 Fax: (204)474-7609 From cisse at cisse2009.org Sun Oct 11 22:02:06 2009 From: cisse at cisse2009.org (CISSE 2009) Date: Sun, 11 Oct 2009 20:02:06 -0700 Subject: [RndTbl] CISSE 2009 - Paper Submission Deadline Extended to October 26, 2009. Message-ID: <200910120312.n9C3CAFs014065@lisa.muug.mb.ca> Dear Colleagues, Due to numerous deadline extension requests from potential CISSE 2009 authors, the CISSE organizing committee has decided to extend the paper submission deadline to 10/26/2009. Please note that this is a hard deadline, so that the technical committees can perform their paper reviewing duties in a timely manner. If you received this email in error, please forward it to the appropriate department at your institution. If you wish to unsubscribe please follow the unsubscribe link at bottom of the email. Please do not reply to this message. If you need to contact us please email us at info at cisse2009.org ********************************************************************* * The Fifth International Joint Conferences on Computer, * * Information, and Systems Sciences, and Engineering (CISSE 2009) * * * * * * * * http://www.cisse2009.org * * * * * * * ********************************************************************* December 4-12, 2009 Sponsored by the University of Bridgeport Technically co-sponsored by the IEEE Computer Society, Communications Society and Education Society (Connecticut Section) --------------------------------------------------------------------- CONFERENCE OVERVIEW --------------------------------------------------------------------- CISSE 2009 provides a virtual forum for presentation and discussion of the state-of the-art research on computers, information and systems sciences and engineering. CISSE 2009 is the fifth conference of the CISSE series of e-conferences. CISSE is the World's first Engineering/Computing and Systems Research E-Conference. CISSE 2005 was the first high-caliber Research Conference in the world to be completely conducted online in real-time via the internet. CISSE 2005 received 255 research paper submissions and the final program included 140 accepted papers, from more than 45 countries. CISSE 2006 received 691 research paper submissions and the final program included 390 accepted papers, from more than 70 countries. CISSE 2007 received 750 research paper submissions and the final program included 406 accepted papers. A total of 948 paper submissions were received for CISSE 2008 and the final program included 382 accepted papers, from more than 80 countries. The virtual conference will be conducted through the Internet using web-conferencing tools, made available by the conference. Authors will be presenting their PowerPoint, audio or video presentations using web-conferencing tools without the need for travel. Conference sessions will be broadcast to all the conference participants, where session participants can interact with the presenter during the presentation and (or) during the Q&A slot that follows the presentation. This international conference will be held entirely on-line. The accepted and presented papers will be made available and sent to the authors after the conference both on a DVD (including all papers, powerpoint presentations and audio presentations) and as a book publication. Springer, the official publisher for CISSE, published the 2005 proceedings in 2 books and the CISSE 2006, CISSE 2007 and CISSE 2008 proceedings in four books each. Conference participants - authors, presenters and attendees - only need an internet connection and sound available on their computers in order to be able to contribute and participate in this international ground-breaking conference. The on-line structure of this high-quality event will allow academic professionals and industry participants to contribute their work and attend world-class technical presentations based on rigorously refereed submissions, live, without the need for investing significant travel funds or time out of the office. The concept and format of CISSE is ground-breaking. The PowerPoint presentations, final paper manuscripts and time schedule for live presentations over the web are available for two weeks prior to the start of the conference for all registrants, so that the participants can choose the presentations they want to attend and think about questions that they might want to ask. The live audio presentations were also recorded and are part of the permanent CISSE on-line archive - accessible to all registrants- which also includes all the papers, PowerPoint and audio presentations. Potential non-author conference attendees who cannot make the on-line conference dates are encouraged to register, as the entire joint conferences will be archived for future viewing. The CISSE conference audio room provides superb audio even over low speed internet connections, the ability to display PowerPoint presentations, and cross-platform compatibility (the conferencing software runs on Windows, Mac, and any other operating system that supports Java). In addition, the conferencing system allows for an unlimited number of participants, which in turn granted us the opportunity to allow all CISSE participants to attend all presentations, as opposed to limiting the number of available seats for each session. Prospective authors are invited to submit full papers electronically in Microsoft Word or Adobe PDF format through the website of the conference at http://www.cisse2009.org. Accepted papers must be presented in the virtual conference by one of the authors. To submit your paper, please visit http://www.cisse2009.org CISSE 2009 is composed of the following four conferences: International Conference on Systems, Computing Sciences and Software Engineering (SCSS 09) Topics: Grid Computing, Internet-based Computing Models, Resource Discovery, Programming Models and tools, e-Science and Virtual Instrumentation, Biometric Authentication, Computers for People of Special Needs, Human Computer Interaction, Information and Knowledge Engineering, Algorithms, Parallel and Distributed processing, Modeling and Simulation, Services and Applications, Embedded Systems and Applications, Databases, Programming Languages, Signal Processing Theory and Methods, Signal Processing for Communication, Signal Processing Architectures and Implementation, Information Processing, Geographical Information Systems, Object Based Software Engineering, Parallel and Distributed Computing, Real Time Systems, Multiprocessing, File Systems and I/O, Kernel and OS Structures. International Conference on Telecommunications and Networking (TeNe 09) Topics: Optical Networks and Switching, Computer Networks, Network architectures and Equipment, Access Technologies, Telecommunication Technology, Coding and Modulation technique, Modeling and Simulation, Spread Spectrum and CDMA Systems, OFDM technology, Space-time Coding, Ultra Wideband Communications, Medium Access Control, Spread Spectrum, Wireless LAN: IEEE 802.11, HIPERLAN, Bluetooth, Cellular Wireless Networks, Cordless Systems and Wireless Local Loop, Mobile Network Layer, Mobile Transport Layer, Support for Mobility, Conventional Encryption and Message Confidentiality, Block Ciphers Design Principles, Block Ciphers Modes of Operation, Public-Key Cryptography and Message Authentication, Authentication Application, Stenography, Electronic Mail Security, Web Security, IP Security, Firewalls, Computer Forensics. International Conference on Engineering Education, Instructional Technology, Assessment, and E-learning (EIAE 09) Topics: Instructional Design, Accreditation, Curriculum Design, Educational Tools, 2-2-2 Platforms, Teaching Capstone Design, Teaching Design at the Lower Levels, Design and Development of e-Learning tools, Assessment Methods in Engineering, Development and Implementation of E-learning tools, Ethics in Education, Economical and Social Impacts of E-learning. International Conference on Industrial Electronics, Technology & Automation (IETA 09) Topics: Advanced and Distributed Control Systems, Intelligent Control Systems (NN, FL, GA, .etc), Expert Systems, Man Machine Interaction, Data Fusion, Factory Automation, Robotics, Motion Control, Machine Vision, MEMS Sensors and Actuators, Sensors Fusion, Power Electronics, High Frequency Converters, Motors and Drives, Power Converters, Power Devices and Components, Electric Vehicles and Intelligent Transportation, Process Automation, Factory Communication, Manufacturing Information System Advances in Manufacturing Systems, Industrial Applications of Multi Media, Intelligent Systems Instrumentation, Industrial Instrumentation, Modeling and Simulation, Signal Processing, Image and Data Processing, VR and Parallel systems.. Paper Submission ================= Prospective authors are invited to submit full papers electronically in Microsoft Word or Adobe PDF format through the website of the conference at http://www.cisse2009.org. Accepted papers must be presented in the virtual conference by one of the authors. To submit your paper, visit http://www.cisse2009.org New Paper submission Deadline: October 26th, 2009 Notification of Acceptance: November 12th, 2009 Final Manuscript and Registration: November 26th, 2009 ------------------------------------------------------------------------ S. Patel CISSE 2009 Support Team University of Bridgeport e-mail: info at cisse2009.org Bridgeport, CT 06604, U.S.A. http://www.cisse2009.org ------------------------------------------------------------------------ Click here on http://server1.streamsend.com/streamsend/unsubscribe.php?cd=3326&md=362&ud=04c04e84971efe5de4fe7c685381812c to update your profile or Unsubscribe From grdetil at scrc.umanitoba.ca Fri Oct 23 14:39:00 2009 From: grdetil at scrc.umanitoba.ca (Gilles Detillieux) Date: Fri, 23 Oct 2009 14:39:00 -0500 Subject: [RndTbl] Linus Torvalds at Windows 7 booth in Tokyo Message-ID: <4AE20654.5060902@scrc.umanitoba.ca> Just thought this would be of interest to the group... http://picasaweb.google.com/cschlaeger/JapanLinuxSymposium#5395358413061926434 -- Gilles R. Detillieux E-mail: Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/ Dept. Physiology, U. of Manitoba Winnipeg, MB R3E 0J9 (Canada) From montanaq at gmail.com Tue Oct 27 14:00:28 2009 From: montanaq at gmail.com (Montana Quiring) Date: Tue, 27 Oct 2009 14:00:28 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD Message-ID: Hello, I want to backup a few Fedora serves and I'm wondering what Open Source software people have used and would recommend for taking a snapshot of a the complete system and have it save to an external USB hard drive. -Montana Blog: http://montanaquiring.info My Friend Feed: http://friendfeed.com/antikx iPhone/Touch Apps I have bought: http://appshopper.com/feed/user/antikx/myapps -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20091027/efb3f385/attachment.html From john at johnlange.ca Tue Oct 27 14:44:49 2009 From: john at johnlange.ca (John Lange) Date: Tue, 27 Oct 2009 14:44:49 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: References: Message-ID: <1256672689.5410.20.camel@vandium.darkcore.net> On Tue, 2009-10-27 at 14:00 -0500, Montana Quiring wrote: > Hello, > > I want to backup a few Fedora serves and I'm wondering what Open > Source software people have used and would recommend for taking a > snapshot of a the complete system and have it save to an external USB > hard drive. Personally I like "dd" for images. dd if=/dev/xxx | gzip > backup.img.gz -- John Lange http://www.johnlange.ca From kel at kelweb.ca Tue Oct 27 14:50:54 2009 From: kel at kelweb.ca (kel at kelweb.ca) Date: Tue, 27 Oct 2009 19:50:54 +0000 Subject: [RndTbl] Open Source Backup/cloning to USB HDD Message-ID: <1398768633-1256673051-cardhu_decombobulator_blackberry.rim.net-182039277-@bda224.bisx.prod.on.blackberry> Clonezilla (GPL) is awesome. The live CD supports backup to network shares as well as transfer over ssh. Kelly ------Original Message------ From: John Lange Sender: roundtable-bounces at muug.mb.ca To: roundtable at muug.mb.ca Subject: Re: [RndTbl] Open Source Backup/cloning to USB HDD Sent: Oct 27, 2009 2:44 PM On Tue, 2009-10-27 at 14:00 -0500, Montana Quiring wrote: > Hello, > > I want to backup a few Fedora serves and I'm wondering what Open > Source software people have used and would recommend for taking a > snapshot of a the complete system and have it save to an external USB > hard drive. Personally I like "dd" for images. dd if=/dev/xxx | gzip > backup.img.gz -- John Lange http://www.johnlange.ca _______________________________________________ Roundtable mailing list Roundtable at muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable From gedetil at cs.umanitoba.ca Tue Oct 27 14:53:27 2009 From: gedetil at cs.umanitoba.ca (Gilbert E. Detillieux) Date: Tue, 27 Oct 2009 14:53:27 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: References: Message-ID: <4AE74FB7.90705@cs.umanitoba.ca> On 2009-10-27 14:00, Montana Quiring wrote: > I want to backup a few Fedora serves and I'm wondering what Open Source > software people have used and would recommend for taking a snapshot of a > the complete system and have it save to an external USB hard drive. I'm using the Amanda backup software, which is part of the Fedora distribution. It's mostly meant for doing backups of multiple clients (over the network) to tape drives/stackers/libraries, but you can also set up a virtual tape device consisting of a set of disk directories. I use that to back up to external disks over various types of interfaces (mostly Firewire). You didn't specify if there are any particular restrictions on the data format you'll be saving to the external drive, but with Amanda, you'd basically be limited to their archive format, where each file has a particular header, followed by the archive data, which can be in either native "dump" format or in GNU tar format, with optional compression. Note that Amanda is really optimized for doing regularly scheduled backups, with periodic full dumps and more frequent incremental dumps. That means, that on a restore, you might have to pull data from multiple archives, just for one file system. If you're looking for something that gives you a complete system image to make "bare metal" restores or cloning easier, this is probably not the best solution. -- Gilbert E. Detillieux E-mail: Dept. of Computer Science Web: http://www.cs.umanitoba.ca/~gedetil/ University of Manitoba Phone: (204)474-8161 Winnipeg MB CANADA R3T 2N2 Fax: (204)474-7609 From grdetil at scrc.umanitoba.ca Tue Oct 27 14:59:42 2009 From: grdetil at scrc.umanitoba.ca (Gilles Detillieux) Date: Tue, 27 Oct 2009 14:59:42 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: <1398768633-1256673051-cardhu_decombobulator_blackberry.rim.net-182039277-@bda224.bisx.prod.on.blackberry> References: <1398768633-1256673051-cardhu_decombobulator_blackberry.rim.net-182039277-@bda224.bisx.prod.on.blackberry> Message-ID: <4AE7512E.4080807@scrc.umanitoba.ca> For file-by-file backups, I like to use the rsync command. E.g.: rsync -a -H -X -x --delete /home/. /mnt/backup/home/. It's very quick and copies over only what's changed (based on file modtimes and/or sizes). It also works well over ssh. On 27/10/2009 2:50 PM, kel at kelweb.ca wrote: > Clonezilla (GPL) is awesome. The live CD supports backup to network shares as well as transfer over ssh. > Kelly > ------Original Message------ > From: John Lange > Sender: roundtable-bounces at muug.mb.ca > To: roundtable at muug.mb.ca > Subject: Re: [RndTbl] Open Source Backup/cloning to USB HDD > Sent: Oct 27, 2009 2:44 PM > > On Tue, 2009-10-27 at 14:00 -0500, Montana Quiring wrote: >> Hello, >> >> I want to backup a few Fedora serves and I'm wondering what Open >> Source software people have used and would recommend for taking a >> snapshot of a the complete system and have it save to an external USB >> hard drive. > > Personally I like "dd" for images. > > dd if=/dev/xxx | gzip > backup.img.gz > -- Gilles R. Detillieux E-mail: Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/ Dept. Physiology, U. of Manitoba Winnipeg, MB R3E 0J9 (Canada) From montanaq at gmail.com Tue Oct 27 15:26:26 2009 From: montanaq at gmail.com (Montana Quiring) Date: Tue, 27 Oct 2009 15:26:26 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> Message-ID: This is from Clonezilla page: Based on Partimage , ntfsclone, partclone , and dd to clone partition. However, clonezilla, containing some other programs, can save and restore not only partitions, but also a whole disk. Thanks for the feedback everyone! -Montana Blog: http://montanaquiring.info My Friend Feed: http://friendfeed.com/antikx iPhone/Touch Apps I have bought: http://appshopper.com/feed/user/antikx/myapps On Tue, Oct 27, 2009 at 3:24 PM, Adam Thompson wrote: > I've used PartImage to great success for all manner of imaging. Excellent > replacement for Symantec/Norton Ghost. I think this is tool used in > CloneZilla, not positive though. SysRescCD bootable image includes this > tool... If you either a) have enough RAM (~765Mb) or b) have another burner > that you *didn't* boot from, it can burn bootable CDs or DVDs. > > -Adam > > -----Original Message----- > From: Montana Quiring > Date: Tue, 27 Oct 2009 14:00:28 > To: MUUG Roundtable > Subject: [RndTbl] Open Source Backup/cloning to USB HDD > > _______________________________________________ > Roundtable mailing list > Roundtable at muug.mb.ca > http://www.muug.mb.ca/mailman/listinfo/roundtable > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20091027/d6fa749c/attachment.html From athompso at athompso.net Tue Oct 27 15:24:13 2009 From: athompso at athompso.net (Adam Thompson) Date: Tue, 27 Oct 2009 20:24:13 +0000 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: References: Message-ID: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> I've used PartImage to great success for all manner of imaging. Excellent replacement for Symantec/Norton Ghost. I think this is tool used in CloneZilla, not positive though. SysRescCD bootable image includes this tool... If you either a) have enough RAM (~765Mb) or b) have another burner that you *didn't* boot from, it can burn bootable CDs or DVDs. -Adam -----Original Message----- From: Montana Quiring Date: Tue, 27 Oct 2009 14:00:28 To: MUUG Roundtable Subject: [RndTbl] Open Source Backup/cloning to USB HDD _______________________________________________ Roundtable mailing list Roundtable at muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable From john at johnlange.ca Tue Oct 27 15:36:02 2009 From: john at johnlange.ca (John Lange) Date: Tue, 27 Oct 2009 15:36:02 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> Message-ID: <1256675762.5410.59.camel@vandium.darkcore.net> Just out of curiosity, what is it that these various programs that people have recommended do with regard to imaging that "dd" doesn't? I've never looked at any of these in any great detail but so far as I can see they all add complexity while reducing functionality. Image a disk? dd Backup a file system? rsync to tape? tar to remote? ssh (or scp) scheduled? put any of the above in cron That's all you need. What am I missing? Regards, -- John Lange http://www.johnlange.ca On Tue, 2009-10-27 at 15:26 -0500, Montana Quiring wrote: > This is from Clonezilla page: > Based on Partimage, ntfsclone, partclone, and dd to clone partition. > However, clonezilla, containing some other programs, can save and > restore not only partitions, but also a whole disk. > > Thanks for the feedback everyone! From montanaq at gmail.com Tue Oct 27 15:39:27 2009 From: montanaq at gmail.com (Montana Quiring) Date: Tue, 27 Oct 2009 15:39:27 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: <1256675762.5410.59.camel@vandium.darkcore.net> References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> <1256675762.5410.59.camel@vandium.darkcore.net> Message-ID: probably nothing extra except have a GUI. I will probably be showing someone that isn't super comfortable with CLI's to do a backup/restore, so I would rather them have a GUI. -Montana Blog: http://montanaquiring.info My Friend Feed: http://friendfeed.com/antikx iPhone/Touch Apps I have bought: http://appshopper.com/feed/user/antikx/myapps On Tue, Oct 27, 2009 at 3:36 PM, John Lange wrote: > Just out of curiosity, what is it that these various programs that > people have recommended do with regard to imaging that "dd" doesn't? > > I've never looked at any of these in any great detail but so far as I > can see they all add complexity while reducing functionality. > > Image a disk? dd > > Backup a file system? rsync > > to tape? tar > > to remote? ssh (or scp) > > scheduled? put any of the above in cron > > That's all you need. What am I missing? > > Regards, > -- > John Lange > http://www.johnlange.ca > > On Tue, 2009-10-27 at 15:26 -0500, Montana Quiring wrote: > > This is from Clonezilla page: > > Based on Partimage, ntfsclone, partclone, and dd to clone partition. > > However, clonezilla, containing some other programs, can save and > > restore not only partitions, but also a whole disk. > > > > Thanks for the feedback everyone! > > > _______________________________________________ > Roundtable mailing list > Roundtable at muug.mb.ca > http://www.muug.mb.ca/mailman/listinfo/roundtable > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20091027/73f87cde/attachment-0001.html From athompso at athompso.net Tue Oct 27 16:58:55 2009 From: athompso at athompso.net (Adam Thompson) Date: Tue, 27 Oct 2009 21:58:55 +0000 Subject: [RndTbl] Open Source Backup/cloning to USB HDD Message-ID: <798565708-1256680713-cardhu_decombobulator_blackberry.rim.net-867254007-@bda467.bisx.prod.on.blackberry> PartImage (and similar & derivative pkgs) add intelligent functionality like volume spanning (back up an 80Gb volume to 10 8Gb DVD-Rs, or to 2Gb files on a FAT filesystem), only backing up used sectors (particularly helpful on drives with tons of free disk space), unicast and/or multicast over-the-wire (for when you can't easily mount the remote filesystem), and automatic partition re-sizing (transfer an image of an 80Gb HDD onto a 300Gb HDD or a 40Gb HDD [as long as there's enough room]). The main advantages come with scenarios *other* than simple disk replacement or cloning to an identical disk, esp. disaster-recovery archiving, mass cloning, etc. -Adam ------Original Message------ From: Lange, John Sender: roundtable-bounces at muug.mb.ca To: Roundtable, MUUG Subject: Re: [RndTbl] Open Source Backup/cloning to USB HDD Sent: Oct 27, 2009 15:36 Just out of curiosity, what is it that these various programs that people have recommended do with regard to imaging that "dd" doesn't? I've never looked at any of these in any great detail but so far as I can see they all add complexity while reducing functionality. Image a disk? dd Backup a file system? rsync to tape? tar to remote? ssh (or scp) scheduled? put any of the above in cron That's all you need. What am I missing? Regards, -- John Lange http://www.johnlange.ca On Tue, 2009-10-27 at 15:26 -0500, Montana Quiring wrote: > This is from Clonezilla page: > Based on Partimage, ntfsclone, partclone, and dd to clone partition. > However, clonezilla, containing some other programs, can save and > restore not only partitions, but also a whole disk. > > Thanks for the feedback everyone! _______________________________________________ Roundtable mailing list Roundtable at muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable From ummar143 at gmail.com Tue Oct 27 18:37:14 2009 From: ummar143 at gmail.com (Dan Martin) Date: Tue, 27 Oct 2009 18:37:14 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: <1256675762.5410.59.camel@vandium.darkcore.net> References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> <1256675762.5410.59.camel@vandium.darkcore.net> Message-ID: <74f2825b0910271637j4d0c5ed3gd88f8177feee33a0@mail.gmail.com> I have used dd for copying drives or partitions in the past, in order to have a clone ready to swap in in the event of a failure. I found it extremely useful, much faster than some cheap commercial software. One problem I have encountered is IO errors, especially when copying large volumes. The 'noerr' option allows the copy to complete, but of course does not correct the problem. I have had one or two errors on every second drive. For my Mac, I have some commercial software that can find the error and what file (if any) it affects. It can report the error to the HSF+ file system, so that no other files use that sector. I can't find any material that explains how this works -- what I surmise is that the drive firmware hides defective blocks. Obviously it does not hide them all, or at least not instantly - so the file system tracks bad blocks that the firmware has missed in order to avoid using them in files. Using dd circumvents the file system, so there is a risk of a corrupt file on the target drive/partition. The drives seem OK in every other respect, and software that monitors the status of the drives does not indicate any impending failure. I am guessing, based on a small sample, that this is a 'normal' condition. On Tue, Oct 27, 2009 at 3:36 PM, John Lange wrote: > Just out of curiosity, what is it that these various programs that > people have recommended do with regard to imaging that "dd" doesn't? > > I've never looked at any of these in any great detail but so far as I > can see they all add complexity while reducing functionality. > > Image a disk? dd > > Backup a file system? rsync > > to tape? tar > > to remote? ssh (or scp) > > scheduled? put any of the above in cron > > That's all you need. What am I missing? > > Regards, > -- > John Lange > http://www.johnlange.ca > > On Tue, 2009-10-27 at 15:26 -0500, Montana Quiring wrote: > > This is from Clonezilla page: > > Based on Partimage, ntfsclone, partclone, and dd to clone partition. > > However, clonezilla, containing some other programs, can save and > > restore not only partitions, but also a whole disk. > > > > Thanks for the feedback everyone! > > > _______________________________________________ > Roundtable mailing list > Roundtable at muug.mb.ca > http://www.muug.mb.ca/mailman/listinfo/roundtable > -- Dan Martin, MD GP Hospital Practitioner Computer Scientist ummar143 at shaw.ca (204) 831-1746 answering machine always on -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20091027/9ea99f56/attachment.html From athompso at athompso.net Tue Oct 27 19:18:24 2009 From: athompso at athompso.net (Adam Thompson) Date: Tue, 27 Oct 2009 19:18:24 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: <74f2825b0910271637j4d0c5ed3gd88f8177feee33a0@mail.gmail.com> References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> <1256675762.5410.59.camel@vandium.darkcore.net> <74f2825b0910271637j4d0c5ed3gd88f8177feee33a0@mail.gmail.com> Message-ID: <8ce1230f0910271718na7910c8x47c2f5e94ad4c80@mail.gmail.com> Unfortunately, based on how the drives are *designed* to operate, seeing any bad sectors at all is NOT NORMAL and probably indicates upcoming failure of the drive. All modern drives (IDE, SATA, SCSI & SAS, anyway) reserve an entire track (or more) for bad-sector remapping. The firmware automatically remaps any sector that requires more than X number of retries to read or write successfully. This is done transparently to the user, operating system, ATAPI controller, etc. - it's handled 100% internally. IF the remapping operation fails, then - and only then - will the read or write command fail and the bad sector will be reported. The main reason for sector remapping operations to fail is that all of the reserved area is used up. Sometimes there are many bad sectors in the vendor-reserved area, which dramatically decreases the number of successful remapping operations that can take place; sometimes it's simply not possible to read the contents of the old, failed, sector no matter how many retries; there are a few other reasons that vary from one manufacturer to the next. But, basically, if the OS is reporting bad sectors, it's time to replace the drive because the drive can no longer perform its own defect management. If you use a tool to examine the drive's SMART response, you'll often see a fairly low number of allowable "bad sectors" - this, too, varies from one manufacturer to the next - because each manufacturer does allow (for warranty purposes) a larger number of sectors to fail than are guaranteed to be remapped successfully. Use a SMART-aware tool to invoke the drive's "Long" self-test, and if necessary, return the drive for repair or replace it. Some spurious read errors may be considered normal; sometimes a drive will attempt to remap a bad block, fail to do so and report the error, then try again and succeed the next time that sector is accessed. It may be considered "normal" that if your drive comes with a one-year warranty, then after that one year, you will gradually develop more and more bad sectors. So any system that isn't brand-new is likely to have some bad sectors. Outside the warranty period, there's no "rule of thumb" for how fast those sectors crop up - so even if you have a two-year-old system, you might already be seeing bad sectors beyond the drive's capability to remap. Note that many modern file systems do not provide any reasonable way to dynamically grow the bad sectors list any more. I would nominate ext2 and ext3 in that list as well as HFS+ and NTFS... not that it's impossible, it's just not reasonably easy for a user to do so. For that matter, I can't think of a single modern FS other than FAT32 where it *is* reasonably easy! Bottom line: bad sectors are bad. You shouldn't see them on modern hard disks. But you probably will. And yes, they really do mean your drive is going bad - but it might still have many usable years left. On Tue, Oct 27, 2009 at 18:37, Dan Martin wrote: > One problem I have encountered is IO errors, especially when copying large > volumes. The 'noerr' option allows the copy to complete, but of course does > not correct the problem. I have had one or two errors on every second > drive. > [...] I can't find any material that explains how this works -- what I surmise is > that the drive firmware hides defective blocks. Obviously it does not hide > them all, or at least not instantly - so the file system tracks bad blocks > that the firmware has missed in order to avoid using them in files. Using > dd circumvents the file system, so there is a risk of a corrupt file on the > target drive/partition. > [...] > The drives seem OK in every other respect, and software that monitors the > status of the drives does not indicate any impending failure. I am > guessing, based on a small sample, that this is a 'normal' condition. -- -Adam Thompson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20091027/54e8086b/attachment.html From athompso at athompso.net Tue Oct 27 21:38:19 2009 From: athompso at athompso.net (Adam Thompson) Date: Tue, 27 Oct 2009 21:38:19 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: <8ce1230f0910271718na7910c8x47c2f5e94ad4c80@mail.gmail.com> References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> <1256675762.5410.59.camel@vandium.darkcore.net> <74f2825b0910271637j4d0c5ed3gd88f8177feee33a0@mail.gmail.com> <8ce1230f0910271718na7910c8x47c2f5e94ad4c80@mail.gmail.com> Message-ID: <8ce1230f0910271938i7a76b6dah5dbdc51bdd0eaa56@mail.gmail.com> On Tue, Oct 27, 2009 at 19:18, Adam Thompson wrote: > > All modern drives (IDE, SATA, SCSI & SAS, anyway) reserve an entire track > (or more) for bad-sector remapping. The firmware automatically remaps any > sector that requires more than X number of retries to read or write > successfully. This is done transparently to the user, operating system, > ATAPI controller, etc. - it's handled 100% internally. > Updating my own information, it seems that many drives only attempt to reallocate bad sectors on WRITE, and *not* on read. The *destructive* SMART "Long" test causes every sector to be re-written, which will accomplish this goal, but at the cost of losing all your data. Windows' ScanDisk program (or CHKDSK, under some versions of Windows) can do a non-destructive surface scan that only tests READS. Gibson Research (GRC)'s SpinRite (http://www.grc.com/spinrite.htm) is the only program that I know and would recommend, that I know can do NON-destructive full-disk read-write testing. It has several limitations, including its US$90 price, but Steve Gibson "wrote the book" on HDD testing and it shows. FYI, if you think $90 is too much money, consider that Steve (the author) only writes in one programming language - x86 assembly! Reportedly, some vendor diagnostics can do non-destructive write testing, but the current versions of Seagate SeaTools and WD DataLifeguard do not appear to have this functionality, and Fujitsu, HGST, Toshiba, and Samsung don't produce drive test utilities [any more]. It would be fairly simple to implement something that does this in Linux with a shell script, something that looped over X megabytes of space, dd(1)'d 1MB of data at a time to a temporary file, dd(1)'d it right back to the drive at offset X. The implementation is left as an exercise for the reader :-). -- -Adam Thompson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20091027/467de075/attachment-0001.html From montanaq at gmail.com Tue Oct 27 22:02:51 2009 From: montanaq at gmail.com (Montana Quiring) Date: Tue, 27 Oct 2009 22:02:51 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: <8ce1230f0910271938i7a76b6dah5dbdc51bdd0eaa56@mail.gmail.com> References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> <1256675762.5410.59.camel@vandium.darkcore.net> <74f2825b0910271637j4d0c5ed3gd88f8177feee33a0@mail.gmail.com> <8ce1230f0910271718na7910c8x47c2f5e94ad4c80@mail.gmail.com> <8ce1230f0910271938i7a76b6dah5dbdc51bdd0eaa56@mail.gmail.com> Message-ID: > Gibson Research (GRC)'s SpinRite (http://www.grc.com/spinrite.htm) is the > only program that I know and would recommend, that I know can do > NON-destructive full-disk read-write testing. It has several limitations, > including its US$90 price, but Steve Gibson "wrote the book" on HDD testing > and it shows. FYI, if you think $90 is too much money, consider that Steve > (the author) only writes in one programming language - x86 assembly! > > Just don't mention SpinRite around Mr. Donovan. *wink* -Montana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20091027/f6ae79f9/attachment.html From peter at pogma.com Wed Oct 28 01:10:47 2009 From: peter at pogma.com (Peter O'Gorman) Date: Wed, 28 Oct 2009 01:10:47 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: <8ce1230f0910271938i7a76b6dah5dbdc51bdd0eaa56@mail.gmail.com> References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> <1256675762.5410.59.camel@vandium.darkcore.net> <74f2825b0910271637j4d0c5ed3gd88f8177feee33a0@mail.gmail.com> <8ce1230f0910271718na7910c8x47c2f5e94ad4c80@mail.gmail.com> <8ce1230f0910271938i7a76b6dah5dbdc51bdd0eaa56@mail.gmail.com> Message-ID: <4AE7E067.5070603@pogma.com> On 10/27/2009 09:38 PM, Adam Thompson wrote: > reallocate bad sectors on WRITE, and *not* on read. The *destructive* > SMART "Long" test causes every sector to be re-written, which will I don't believe the SMART long test is destructive, why do you think so? Peter -- Peter O'Gorman http://pogma.com From athompso at athompso.net Wed Oct 28 01:28:28 2009 From: athompso at athompso.net (Adam Thompson) Date: Wed, 28 Oct 2009 01:28:28 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: <4AE7E067.5070603@pogma.com> References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> <1256675762.5410.59.camel@vandium.darkcore.net> <74f2825b0910271637j4d0c5ed3gd88f8177feee33a0@mail.gmail.com> <8ce1230f0910271718na7910c8x47c2f5e94ad4c80@mail.gmail.com> <8ce1230f0910271938i7a76b6dah5dbdc51bdd0eaa56@mail.gmail.com> <4AE7E067.5070603@pogma.com> Message-ID: <8ce1230f0910272328g52ff20dcv95c87abc9365c736@mail.gmail.com> On Wed, Oct 28, 2009 at 01:10, Peter O'Gorman wrote: > On 10/27/2009 09:38 PM, Adam Thompson wrote: > > reallocate bad sectors on WRITE, and *not* on read. The *destructive* > > SMART "Long" test causes every sector to be re-written, which will > I don't believe the SMART long test is destructive, why do you think so? > Sorry, I wasn't clear enough in my original posting. The regular SMART "Long" test is *not* destructive. Some HDDs - there's no pattern than I can discern to tell which, other than reading the documentation - offer an additional, *non-standard* destructive test. Some of *those* HDDs appear to invoke the same function as "low-level format", since both are essentially equivalent processes. Sector remapping takes place as usual during either command. Generally speaking, SCSI, FC and SAS devices should always support a low-level format command, per ANSI T13. IDE and SATA devices may or may not support a low-level format command. The SMART specification (per ANSI T13) is extensible, and specifies a *minimum* level of functionality; some vendors - at least in certain models - implement more than minimum required functionality. It seems to me that models sold as OEM equipment to server and storage appliance vendors often implement extra functionality, probably at the request of vendors like NetApp, EMC, Dell, HP, etc. Since most storage vendors now offer cheap(er) SATA storage options for their arrays, some so-called "desktop" models will also benefit from this. The only way to be certain is to read the technical documentation for your particular drive, which will tell you precisely what commands the drive supports. I'm not talking about the user manual; note that some drives do not appear to have any such documentation - in which case you can probably assume they don't support anything beyond the minimum required feature set. -- -Adam Thompson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20091028/8619280b/attachment.html From wyatt at prairieturtle.ca Thu Oct 29 22:47:18 2009 From: wyatt at prairieturtle.ca (Daryl F) Date: Thu, 29 Oct 2009 22:47:18 -0500 (CDT) Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: <74f2825b0910271637j4d0c5ed3gd88f8177feee33a0@mail.gmail.com> References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> <1256675762.5410.59.camel@vandium.darkcore.net> <74f2825b0910271637j4d0c5ed3gd88f8177feee33a0@mail.gmail.com> Message-ID: To (sometimes) get around dd stopping on the first error or using 'noerr' and getting nothing copied for a sector, I've been using a program called ddrescue ref: http://directory.fsf.org/project/ddrescue/ that does several retries before giving up. It also has some interesting techniques to recover as much as possible from a damaged disks while minimizing further damage. -Daryl On Tue, 27 Oct 2009, Dan Martin wrote: > I have used dd for copying drives or partitions in the past, in order to > have a clone ready to swap in in the event of a failure. I found it > extremely useful, much faster than some cheap commercial software. > > One problem I have encountered is IO errors, especially when copying large > volumes. The 'noerr' option allows the copy to complete, but of course does > not correct the problem. I have had one or two errors on every second > drive. > > For my Mac, I have some commercial software that can find the error and what > file (if any) it affects. It can report the error to the HSF+ file system, > so that no other files use that sector. > > I can't find any material that explains how this works -- what I surmise is > that the drive firmware hides defective blocks. Obviously it does not hide > them all, or at least not instantly - so the file system tracks bad blocks > that the firmware has missed in order to avoid using them in files. Using > dd circumvents the file system, so there is a risk of a corrupt file on the > target drive/partition. > > The drives seem OK in every other respect, and software that monitors the > status of the drives does not indicate any impending failure. I am > guessing, based on a small sample, that this is a 'normal' condition. > > On Tue, Oct 27, 2009 at 3:36 PM, John Lange wrote: > >> Just out of curiosity, what is it that these various programs that >> people have recommended do with regard to imaging that "dd" doesn't? >> >> I've never looked at any of these in any great detail but so far as I >> can see they all add complexity while reducing functionality. >> >> Image a disk? dd >> >> Backup a file system? rsync >> >> to tape? tar >> >> to remote? ssh (or scp) >> >> scheduled? put any of the above in cron >> >> That's all you need. What am I missing? >> >> Regards, >> -- >> John Lange >> http://www.johnlange.ca >> >> On Tue, 2009-10-27 at 15:26 -0500, Montana Quiring wrote: >>> This is from Clonezilla page: >>> Based on Partimage, ntfsclone, partclone, and dd to clone partition. >>> However, clonezilla, containing some other programs, can save and >>> restore not only partitions, but also a whole disk. >>> >>> Thanks for the feedback everyone! >> >> >> _______________________________________________ >> Roundtable mailing list >> Roundtable at muug.mb.ca >> http://www.muug.mb.ca/mailman/listinfo/roundtable >> > > > > -Daryl From athompso at athompso.net Thu Oct 29 23:08:22 2009 From: athompso at athompso.net (Adam Thompson) Date: Thu, 29 Oct 2009 23:08:22 -0500 Subject: [RndTbl] Open Source Backup/cloning to USB HDD In-Reply-To: References: <618880304-1256675032-cardhu_decombobulator_blackberry.rim.net-1009461190-@bda467.bisx.prod.on.blackberry> <1256675762.5410.59.camel@vandium.darkcore.net> <74f2825b0910271637j4d0c5ed3gd88f8177feee33a0@mail.gmail.com> Message-ID: <8ce1230f0910292108r46996bb8nd70f67b5ebae0dbc@mail.gmail.com> It's fascinating (ok, at least to the ?bergeeks among us) to note that SpinRite and GNU ddrescue have diametrically opposite approaches to data recovery. Yes despite being completely opposite, both approaches are still valid and each will work well in different scenarios. I've been reminded by a few people that SpinRite in particular, while a very useful tool, can actually cause data LOSS in some scenarios - if you choose to purchase and use this (IMHO) extremely useful utility, PLEASE READ THE MANUAL FIRST. And beyond that, if you don't UNDERSTAND the manual, write Steve Gibson and ask (politely) for your money back - because if you don't understand what the manual is talking about, you are likely to use the tool incorrectly and lose more data than you save. Think of SpinRite as a lethal weapon - highly useful in the right situation, but if you don't know how to use it you'll just blow your foot off by accident. Thank you for listening :-) -Adam On Thu, Oct 29, 2009 at 22:47, Daryl F wrote: > To (sometimes) get around dd stopping on the first error or using 'noerr' > and getting nothing copied for a sector, I've been using a program called > ddrescue ref: http://directory.fsf.org/project/ddrescue/ that does several > retries before giving up. It also has some interesting techniques to > recover as much as possible from a damaged disks while minimizing further > damage. > -- -Adam Thompson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20091029/eaed2242/attachment.html