1 Using UDEV to do Persistent storage device naming
2 for large numbers of storage devices
5 Here are some lessons we learned at OSDL recently on how to use UDEV
6 (version 021) to do persistent device naming for lots of storage devices.
7 We used what was available in udev for scsi devices. Here is an outline of
10 Background information
11 a list of resources we needed to get started.
13 what we needed to create the right enviroment (kernel, patches,
15 How udev works to assign persistent storage device names
16 what the documentation didn't tell us.
18 A sanity test we ran to compare with and without persistent naming.
21 BACKGROUND INFORMATION
22 To get started, here are some references. Review the overview articles so
23 that the rest of the information makes sense.
25 Download the latest udev stuff from:
26 http://www.kernel.org/pub/linux/utils/kernel/hotplug/
29 linux-hotplug-devel@lists.sourceforge.net
31 Here is a nice overview article to get started (warning, this is from
32 summer 2003 so many items indicated as "todo" have been done and
33 configuration file name references have sometime changed):
34 http://www.kroah.com/linux/talks/ols_2003_udev_paper/Reprint-Kroah-Hartman-OLS2003.pdf
35 (also included when you download udev)
37 More general info (also included in the udev package):
38 http://kernel.org/pub/linux/utils/kernel/hotplug/udev-FAQ
39 UDEV version 021 Announcement:
40 http://marc.theaimsgroup.com/?l=linux-hotplug-devel&m=107827264803336&w=2
42 "Managing Dynamic Naming":
43 http://lwn.net/Articles/28897/
45 If you are a fan of devfs, whatever you do, don't complain until you read
46 everything you possibly can about udev. This for example:
47 http://kernel.org/pub/linux/utils/kernel/hotplug/udev_vs_devfs
49 You will need to create udev.rules to supply consistent names. (See
50 etc/udev/udev.rules in the download). This article gives you some
51 background about udev.rules, but avoids describing the "PROGRAM" key which
52 is needed for our work. Read it for background: writing udev rules
53 (current as of udev 018)
54 http://www.reactivated.net/udevrules.php
57 bk://kernel.bkbits.net/gregkh/udev
59 Libsysfs used to get sysfs information):
60 http://www-124.ibm.com/linux/papers/libsysfs/libsysfs-linuxconfau2004.pdf
62 UDEV works using the way hotplug events are handled by the kernel.
63 Several overview articles about hotplug include:
65 http://lwn.net/Articles/52621/
67 http://linux-hotplug.sourceforge.net/
69 Gentoo centric install info:
70 http://webpages.charter.net/decibelshelp/LinuxHelp_UDEVPrimer.html
72 rpms built against Red Hat FC2-test1 may be available at:
73 http://kernel.org/pub/linux/utils/kernel/hotplug/udev-021-1.i386.rpm
75 with the source rpm at:
76 http://kernel.org/pub/linux/utils/kernel/hotplug/udev-021-1.src.rpm
82 Here is a brief checklist of what you need on your system for this to
85 Kernel must be a 2.6 kernel
87 Must use CONFIG_HOTPLUG kernel config option, since the solution is based
88 on hotplug capabilities.
90 To test more than 256 scsi devices you need a patch to the scsi driver to
91 support that many (available from IBM or SuSE). To see the patch we used,
93 http://developer.osdl.org/maryedie/DCL/PSDN/lotsofdisks.patch
95 Your storage device must support (via the driver) a unique identifier for
96 persistent device naming. (Adaptec RAID device does not, for example.)
98 Your device driver must support sysfs (new in 2.6 kernel). This is already
99 done for scsi devices and most if not all block devices.
101 A program (scsi_id) exists in the udev download (extras/scsi_id/scsi_id.c)
102 for scsi devices. It can read the identifier and is needed for persistent
106 HOW UDEV WORKS TO ASSIGN PERSISTENT NAMES:
108 There are three places where device information is stored that udev
110 (1) /sys maintained by sysfs
111 (2) /etc/udev/udev.rules - where you can store the identifier to NAME
113 (3) The udevdb, that keeps track the valid system configuration.
114 It is constructed at boot time and updated with configuration changes.
116 The persistent names are kept (at least this is one way to do it) in
117 udev.rules (uuid and NAME), one entry per device. If you want to initially
118 give your 1000 disk devices a default name and then make sure those names
119 are preserved, here is how :
121 Start with no special entry in udev.rules when do you an initial boot of
122 your system with disks in place. Udev will assign default names (there
123 are ways to control what you want for default too).
125 Once the names are assigned, use a script supplied for scsi devices -
126 udev-021/extras/scsi_id/gen_scsi_id_udev_rules.sh to generate the lines
127 needed for udev.rules, one per device. Each line indicates the identifier
128 and the NAME it was assigned. You could optionally create this manually if
129 you prefer other names .
131 [example entries in udev.rules for scsi disks]
132 BUS="scsi", PROGRAM="scsi_id", RESULT="<uuid1>",NAME="<name1>"
133 BUS="scsi", RESULT="<uuid2>",NAME="<name2>"
135 BUS="scsi", RESULT="<uuid1000>",NAME="<name1000>"
137 (The actual file we used is the file udev.rules_1000_scsi_debug in this
140 Upon reboot, for each device a hotplug event occurs. The udev.rules file
141 is scanned looking for the device type (BUS) in this case for "scsi". The
142 first entry generated by the above program references a PROGRAM in the key
143 field (scsi_id) which is called to probe the device and determine the
144 unique identifier. sysfs is used to determine the major/minor number for
145 the device. The result of the program execution (the uuid) is compared
146 with the RESULT entry in the same udev.rules line.
148 - If it matches, then the NAME entered on this line is used. The uuid and
149 major/minor number is saved in the udevdb (newly recreated upon boot).
150 That device is created in /udev (the target directory name is configurable)
151 with the assigned NAME.
153 - If it doesn't match, the RESULT (uuid) is preserved for use on the next
154 udev.rules line as long as the bus type (scsi) is the same. So the
155 result (the uuid) is compared on the next line, and the next until a
158 - If no match occurs, the device will be assigned a default name.
160 - The udevdb is updated with the resulting name assignment.
163 Thus if the uuid and names are enumerated, they will be found, assigned,
164 and are therefore permanent.
166 If the device is removed from a live system, a hotplug event occurs, and it
167 is removed from udevdb and the /udev entry disappears.
169 If it is re-inserted at a new location, the udev.rules file is scanned as
170 above. The rule matches again against the uuid, the name in udev.rules
171 is applied again and the /udev name re-appears.
177 Now the question becomes, how much longer does it take to scan the
178 udev.rules table once there are 1000 entries?
180 To test this, we created 1000 "scsi " devices using the scsi debug device
181 driver supplied in the kernel. When this device driver is loaded you can
182 specify how many fake scsi devices to create. There is no real I/O
183 involved but it does respond to some scsi commands. It simulates the uuid
184 by using the device number assigned when the device is created.
186 Then we auto-generated entries into udev.rules with
187 gen_scsi_id_udev_rules.sh. We then removed the devices and reassigned them
188 to simulate a reboot. The delta between assigning defaults and assigning
189 the names enumerated in the udev.rules file was 7 seconds (that's for 1000
192 Scripts utilized the feature (described above) that saves the "RESULT" key
193 after one scsi-id program call for later reference with other udev.rules
194 entries (so only have one PROGRAM key is the moral of the story). If you
195 repeated the PROGRAM key, you would unnecessarily call the program up to
198 The script that creates udev.rules did not work for 1000 drives (the input
199 line is too long). We determined that a patch for this already existed but
200 had not yet been checked in.