<html><body bgcolor="#FFFFFF"><div>Instead if /dev/zero use another deduce to read blindly. There is no guarantee /dev/zero will not produce a sparse file.</div><div><br></div><div>Generate a realistic data file from a sufferer volume (say the boot volume</div><div>or a well used/full USB key or drive). Better data set, better entropy to avoid caching ruining the results.</div><div><br></div><div>I'm interested in the results too!</div><div><br>-- <div>Sean (mobile)</div></div><div><br>On 2011-05-19, at 9:27 PM, "Adam Thompson" <<a href="mailto:athompso@athompso.net">athompso@athompso.net</a>> wrote:<br><br></div><div></div><blockquote type="cite"><div><div class="WordSection1"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Well, I did say there was more than one variable involved! 25MB/sec is a bit slow, but I don’t know how efficiently that LSI chip is at managing bus contention, or how efficient the kernel driver for that chip is. I do know from experience that 8-disk arrays are well into the land of diminishing returns from a speed perspective: RAID-1 on two disks or RAID-10 on four seems to be the sweet spot for speed.<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Once your rebuild has finished, I would recommend doing some throughput tests (both reading and writing) on the array; something perhaps like:<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"># sync; time sh –c ‘dd if=/dev/zero of=/mnt/raidarray/BIGFILE.ZERO bs=1M count=1024;sync’<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">followed by<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"># time dd if=/dev/md0 of=/dev/null bs=1M count=1024<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Those are both very naïve approaches, but should give you a feel for the maximum read and write speeds of your array. I strongly suspect those numbers will be much higher than the raid re-sync rate, again, mostly due to write-flush barriers in the md code.<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I’m interested in knowing how this ends, personally… please let us know.<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">-Adam<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">(P.S. does anyone know how to avoid top-posting in Outlook 2010?)<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p><p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></a></p><div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:roundtable-bounces@muug.mb.ca">roundtable-bounces@muug.mb.ca</a> [mailto:roundtable-bounces@muug.mb.ca] <b>On Behalf Of </b>Kevin McGregor<br><b>Sent:</b> Thursday, May 19, 2011 14:52<br><b>To:</b> Continuation of Round Table discussion<br><b>Subject:</b> Re: [RndTbl] RAID5 rebuild performance<o:p></o:p></span></p></div></div><p class="MsoNormal"><o:p> </o:p></p><p class="MsoNormal" style="margin-bottom:12.0pt">raid6: using algorithm sse2x2 (2883 MB/s). So, 1% of that is reasonable? :-) Oh well, I guess I can wait until tomorrow for the rebuild to finish.<o:p></o:p></p><div><p class="MsoNormal">On Thu, May 19, 2011 at 2:12 PM, Adam Thompson <<a href="mailto:athompso@athompso.net"><a href="mailto:athompso@athompso.net">athompso@athompso.net</a></a>> wrote:<o:p></o:p></p><p class="MsoNormal">Look back in dmesg output for the RAID module speed tests, notice which one was selected, and divide that number by 2. There's your theoretical bottleneck on the CPU.<br>Take the minimum sustained disk/channel/controller throughput, factor in interrupt latency, device driver efficiency, etc. and make a rough guess as to the overall throughput.<br>Consider that md code seems to have a lot of write barriers for safety - so even a rebuild will aspens much of its time waiting for the disk to sync().<br>All in all, I think your numbers are probably reasonable.<br><span style="color:#888888">-Adam</span><o:p></o:p></p><div><div><p class="MsoNormal"><br><br>Kevin McGregor <<a href="mailto:kevin.a.mcgregor@gmail.com"><a href="mailto:kevin.a.mcgregor@gmail.com">kevin.a.mcgregor@gmail.com</a></a>> wrote:<br><br>>I installed Ubuntu Server 10.04.2 LTS AMD64 on a HP ProLiant ML370 G3 (4 x<br>>dual-core/hyperthreaded Xeon 2.66 GHz, 8 GB RAM) and I used the on-board<br>>SCSI controller to manage 8 x 300 GB 15K RPM SCSI drives in a software RAID<br>>5 set up as a 7-drive array with 1 hot-spare drive. All drives are the exact<br>>same model with the same firmware version.<br>><br>>It's currently rebuilding the array (because I just created the array) and<br>>/proc/mdstat is reporting "finish=165.7min speed=25856K/sec". Does that<br>>sound "right" in the sense that it's the right order of magnitude? I though<br>>it should be higher, but I haven't set up such an array before, so I don't<br>>have anything to compare it to.<br>><br>>If it's slow, does anyone have a suggestion for speeding it up?<br>><br>>Kevin<br>><br>>_______________________________________________<br>>Roundtable mailing list<br>><a href="mailto:Roundtable@muug.mb.ca"><a href="mailto:Roundtable@muug.mb.ca">Roundtable@muug.mb.ca</a></a><br>><a href="http://www.muug.mb.ca/mailman/listinfo/roundtable" target="_blank"><a href="http://www.muug.mb.ca/mailman/listinfo/roundtable">http://www.muug.mb.ca/mailman/listinfo/roundtable</a></a><br><br>_______________________________________________<br>Roundtable mailing list<br><a href="mailto:Roundtable@muug.mb.ca"><a href="mailto:Roundtable@muug.mb.ca">Roundtable@muug.mb.ca</a></a><br><a href="http://www.muug.mb.ca/mailman/listinfo/roundtable" target="_blank"><a href="http://www.muug.mb.ca/mailman/listinfo/roundtable">http://www.muug.mb.ca/mailman/listinfo/roundtable</a></a><o:p></o:p></p></div></div></div><p class="MsoNormal"><o:p> </o:p></p></div></div></div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>Roundtable mailing list</span><br><span><a href="mailto:Roundtable@muug.mb.ca">Roundtable@muug.mb.ca</a></span><br><span><a href="http://www.muug.mb.ca/mailman/listinfo/roundtable">http://www.muug.mb.ca/mailman/listinfo/roundtable</a></span><br></div></blockquote></body></html>