autofs-mount-control.txt 18 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406
  1. Miscellaneous Device control operations for the autofs kernel module
  2. ====================================================================
  3. The problem
  4. ===========
  5. There is a problem with active restarts in autofs (that is to say
  6. restarting autofs when there are busy mounts).
  7. During normal operation autofs uses a file descriptor opened on the
  8. directory that is being managed in order to be able to issue control
  9. operations. Using a file descriptor gives ioctl operations access to
  10. autofs specific information stored in the super block. The operations
  11. are things such as setting an autofs mount catatonic, setting the
  12. expire timeout and requesting expire checks. As is explained below,
  13. certain types of autofs triggered mounts can end up covering an autofs
  14. mount itself which prevents us being able to use open(2) to obtain a
  15. file descriptor for these operations if we don't already have one open.
  16. Currently autofs uses "umount -l" (lazy umount) to clear active mounts
  17. at restart. While using lazy umount works for most cases, anything that
  18. needs to walk back up the mount tree to construct a path, such as
  19. getcwd(2) and the proc file system /proc/<pid>/cwd, no longer works
  20. because the point from which the path is constructed has been detached
  21. from the mount tree.
  22. The actual problem with autofs is that it can't reconnect to existing
  23. mounts. Immediately one thinks of just adding the ability to remount
  24. autofs file systems would solve it, but alas, that can't work. This is
  25. because autofs direct mounts and the implementation of "on demand mount
  26. and expire" of nested mount trees have the file system mounted directly
  27. on top of the mount trigger directory dentry.
  28. For example, there are two types of automount maps, direct (in the kernel
  29. module source you will see a third type called an offset, which is just
  30. a direct mount in disguise) and indirect.
  31. Here is a master map with direct and indirect map entries:
  32. /- /etc/auto.direct
  33. /test /etc/auto.indirect
  34. and the corresponding map files:
  35. /etc/auto.direct:
  36. /automount/dparse/g6 budgie:/autofs/export1
  37. /automount/dparse/g1 shark:/autofs/export1
  38. and so on.
  39. /etc/auto.indirect:
  40. g1 shark:/autofs/export1
  41. g6 budgie:/autofs/export1
  42. and so on.
  43. For the above indirect map an autofs file system is mounted on /test and
  44. mounts are triggered for each sub-directory key by the inode lookup
  45. operation. So we see a mount of shark:/autofs/export1 on /test/g1, for
  46. example.
  47. The way that direct mounts are handled is by making an autofs mount on
  48. each full path, such as /automount/dparse/g1, and using it as a mount
  49. trigger. So when we walk on the path we mount shark:/autofs/export1 "on
  50. top of this mount point". Since these are always directories we can
  51. use the follow_link inode operation to trigger the mount.
  52. But, each entry in direct and indirect maps can have offsets (making
  53. them multi-mount map entries).
  54. For example, an indirect mount map entry could also be:
  55. g1 \
  56. / shark:/autofs/export5/testing/test \
  57. /s1 shark:/autofs/export/testing/test/s1 \
  58. /s2 shark:/autofs/export5/testing/test/s2 \
  59. /s1/ss1 shark:/autofs/export1 \
  60. /s2/ss2 shark:/autofs/export2
  61. and a similarly a direct mount map entry could also be:
  62. /automount/dparse/g1 \
  63. / shark:/autofs/export5/testing/test \
  64. /s1 shark:/autofs/export/testing/test/s1 \
  65. /s2 shark:/autofs/export5/testing/test/s2 \
  66. /s1/ss1 shark:/autofs/export2 \
  67. /s2/ss2 shark:/autofs/export2
  68. One of the issues with version 4 of autofs was that, when mounting an
  69. entry with a large number of offsets, possibly with nesting, we needed
  70. to mount and umount all of the offsets as a single unit. Not really a
  71. problem, except for people with a large number of offsets in map entries.
  72. This mechanism is used for the well known "hosts" map and we have seen
  73. cases (in 2.4) where the available number of mounts are exhausted or
  74. where the number of privileged ports available is exhausted.
  75. In version 5 we mount only as we go down the tree of offsets and
  76. similarly for expiring them which resolves the above problem. There is
  77. somewhat more detail to the implementation but it isn't needed for the
  78. sake of the problem explanation. The one important detail is that these
  79. offsets are implemented using the same mechanism as the direct mounts
  80. above and so the mount points can be covered by a mount.
  81. The current autofs implementation uses an ioctl file descriptor opened
  82. on the mount point for control operations. The references held by the
  83. descriptor are accounted for in checks made to determine if a mount is
  84. in use and is also used to access autofs file system information held
  85. in the mount super block. So the use of a file handle needs to be
  86. retained.
  87. The Solution
  88. ============
  89. To be able to restart autofs leaving existing direct, indirect and
  90. offset mounts in place we need to be able to obtain a file handle
  91. for these potentially covered autofs mount points. Rather than just
  92. implement an isolated operation it was decided to re-implement the
  93. existing ioctl interface and add new operations to provide this
  94. functionality.
  95. In addition, to be able to reconstruct a mount tree that has busy mounts,
  96. the uid and gid of the last user that triggered the mount needs to be
  97. available because these can be used as macro substitution variables in
  98. autofs maps. They are recorded at mount request time and an operation
  99. has been added to retrieve them.
  100. Since we're re-implementing the control interface, a couple of other
  101. problems with the existing interface have been addressed. First, when
  102. a mount or expire operation completes a status is returned to the
  103. kernel by either a "send ready" or a "send fail" operation. The
  104. "send fail" operation of the ioctl interface could only ever send
  105. ENOENT so the re-implementation allows user space to send an actual
  106. status. Another expensive operation in user space, for those using
  107. very large maps, is discovering if a mount is present. Usually this
  108. involves scanning /proc/mounts and since it needs to be done quite
  109. often it can introduce significant overhead when there are many entries
  110. in the mount table. An operation to lookup the mount status of a mount
  111. point dentry (covered or not) has also been added.
  112. Current kernel development policy recommends avoiding the use of the
  113. ioctl mechanism in favor of systems such as Netlink. An implementation
  114. using this system was attempted to evaluate its suitability and it was
  115. found to be inadequate, in this case. The Generic Netlink system was
  116. used for this as raw Netlink would lead to a significant increase in
  117. complexity. There's no question that the Generic Netlink system is an
  118. elegant solution for common case ioctl functions but it's not a complete
  119. replacement probably because its primary purpose in life is to be a
  120. message bus implementation rather than specifically an ioctl replacement.
  121. While it would be possible to work around this there is one concern
  122. that lead to the decision to not use it. This is that the autofs
  123. expire in the daemon has become far to complex because umount
  124. candidates are enumerated, almost for no other reason than to "count"
  125. the number of times to call the expire ioctl. This involves scanning
  126. the mount table which has proved to be a big overhead for users with
  127. large maps. The best way to improve this is try and get back to the
  128. way the expire was done long ago. That is, when an expire request is
  129. issued for a mount (file handle) we should continually call back to
  130. the daemon until we can't umount any more mounts, then return the
  131. appropriate status to the daemon. At the moment we just expire one
  132. mount at a time. A Generic Netlink implementation would exclude this
  133. possibility for future development due to the requirements of the
  134. message bus architecture.
  135. autofs Miscellaneous Device mount control interface
  136. ====================================================
  137. The control interface is opening a device node, typically /dev/autofs.
  138. All the ioctls use a common structure to pass the needed parameter
  139. information and return operation results:
  140. struct autofs_dev_ioctl {
  141. __u32 ver_major;
  142. __u32 ver_minor;
  143. __u32 size; /* total size of data passed in
  144. * including this struct */
  145. __s32 ioctlfd; /* automount command fd */
  146. /* Command parameters */
  147. union {
  148. struct args_protover protover;
  149. struct args_protosubver protosubver;
  150. struct args_openmount openmount;
  151. struct args_ready ready;
  152. struct args_fail fail;
  153. struct args_setpipefd setpipefd;
  154. struct args_timeout timeout;
  155. struct args_requester requester;
  156. struct args_expire expire;
  157. struct args_askumount askumount;
  158. struct args_ismountpoint ismountpoint;
  159. };
  160. char path[0];
  161. };
  162. The ioctlfd field is a mount point file descriptor of an autofs mount
  163. point. It is returned by the open call and is used by all calls except
  164. the check for whether a given path is a mount point, where it may
  165. optionally be used to check a specific mount corresponding to a given
  166. mount point file descriptor, and when requesting the uid and gid of the
  167. last successful mount on a directory within the autofs file system.
  168. The union is used to communicate parameters and results of calls made
  169. as described below.
  170. The path field is used to pass a path where it is needed and the size field
  171. is used account for the increased structure length when translating the
  172. structure sent from user space.
  173. This structure can be initialized before setting specific fields by using
  174. the void function call init_autofs_dev_ioctl(struct autofs_dev_ioctl *).
  175. All of the ioctls perform a copy of this structure from user space to
  176. kernel space and return -EINVAL if the size parameter is smaller than
  177. the structure size itself, -ENOMEM if the kernel memory allocation fails
  178. or -EFAULT if the copy itself fails. Other checks include a version check
  179. of the compiled in user space version against the module version and a
  180. mismatch results in a -EINVAL return. If the size field is greater than
  181. the structure size then a path is assumed to be present and is checked to
  182. ensure it begins with a "/" and is NULL terminated, otherwise -EINVAL is
  183. returned. Following these checks, for all ioctl commands except
  184. AUTOFS_DEV_IOCTL_VERSION_CMD, AUTOFS_DEV_IOCTL_OPENMOUNT_CMD and
  185. AUTOFS_DEV_IOCTL_CLOSEMOUNT_CMD the ioctlfd is validated and if it is
  186. not a valid descriptor or doesn't correspond to an autofs mount point
  187. an error of -EBADF, -ENOTTY or -EINVAL (not an autofs descriptor) is
  188. returned.
  189. The ioctls
  190. ==========
  191. An example of an implementation which uses this interface can be seen
  192. in autofs version 5.0.4 and later in file lib/dev-ioctl-lib.c of the
  193. distribution tar available for download from kernel.org in directory
  194. /pub/linux/daemons/autofs/v5.
  195. The device node ioctl operations implemented by this interface are:
  196. AUTOFS_DEV_IOCTL_VERSION
  197. ------------------------
  198. Get the major and minor version of the autofs device ioctl kernel module
  199. implementation. It requires an initialized struct autofs_dev_ioctl as an
  200. input parameter and sets the version information in the passed in structure.
  201. It returns 0 on success or the error -EINVAL if a version mismatch is
  202. detected.
  203. AUTOFS_DEV_IOCTL_PROTOVER_CMD and AUTOFS_DEV_IOCTL_PROTOSUBVER_CMD
  204. ------------------------------------------------------------------
  205. Get the major and minor version of the autofs protocol version understood
  206. by loaded module. This call requires an initialized struct autofs_dev_ioctl
  207. with the ioctlfd field set to a valid autofs mount point descriptor
  208. and sets the requested version number in version field of struct args_protover
  209. or sub_version field of struct args_protosubver. These commands return
  210. 0 on success or one of the negative error codes if validation fails.
  211. AUTOFS_DEV_IOCTL_OPENMOUNT and AUTOFS_DEV_IOCTL_CLOSEMOUNT
  212. ----------------------------------------------------------
  213. Obtain and release a file descriptor for an autofs managed mount point
  214. path. The open call requires an initialized struct autofs_dev_ioctl with
  215. the path field set and the size field adjusted appropriately as well
  216. as the devid field of struct args_openmount set to the device number of
  217. the autofs mount. The device number can be obtained from the mount options
  218. shown in /proc/mounts. The close call requires an initialized struct
  219. autofs_dev_ioct with the ioctlfd field set to the descriptor obtained
  220. from the open call. The release of the file descriptor can also be done
  221. with close(2) so any open descriptors will also be closed at process exit.
  222. The close call is included in the implemented operations largely for
  223. completeness and to provide for a consistent user space implementation.
  224. AUTOFS_DEV_IOCTL_READY_CMD and AUTOFS_DEV_IOCTL_FAIL_CMD
  225. --------------------------------------------------------
  226. Return mount and expire result status from user space to the kernel.
  227. Both of these calls require an initialized struct autofs_dev_ioctl
  228. with the ioctlfd field set to the descriptor obtained from the open
  229. call and the token field of struct args_ready or struct args_fail set
  230. to the wait queue token number, received by user space in the foregoing
  231. mount or expire request. The status field of struct args_fail is set to
  232. the errno of the operation. It is set to 0 on success.
  233. AUTOFS_DEV_IOCTL_SETPIPEFD_CMD
  234. ------------------------------
  235. Set the pipe file descriptor used for kernel communication to the daemon.
  236. Normally this is set at mount time using an option but when reconnecting
  237. to a existing mount we need to use this to tell the autofs mount about
  238. the new kernel pipe descriptor. In order to protect mounts against
  239. incorrectly setting the pipe descriptor we also require that the autofs
  240. mount be catatonic (see next call).
  241. The call requires an initialized struct autofs_dev_ioctl with the
  242. ioctlfd field set to the descriptor obtained from the open call and
  243. the pipefd field of struct args_setpipefd set to descriptor of the pipe.
  244. On success the call also sets the process group id used to identify the
  245. controlling process (eg. the owning automount(8) daemon) to the process
  246. group of the caller.
  247. AUTOFS_DEV_IOCTL_CATATONIC_CMD
  248. ------------------------------
  249. Make the autofs mount point catatonic. The autofs mount will no longer
  250. issue mount requests, the kernel communication pipe descriptor is released
  251. and any remaining waits in the queue released.
  252. The call requires an initialized struct autofs_dev_ioctl with the
  253. ioctlfd field set to the descriptor obtained from the open call.
  254. AUTOFS_DEV_IOCTL_TIMEOUT_CMD
  255. ----------------------------
  256. Set the expire timeout for mounts within an autofs mount point.
  257. The call requires an initialized struct autofs_dev_ioctl with the
  258. ioctlfd field set to the descriptor obtained from the open call.
  259. AUTOFS_DEV_IOCTL_REQUESTER_CMD
  260. ------------------------------
  261. Return the uid and gid of the last process to successfully trigger a the
  262. mount on the given path dentry.
  263. The call requires an initialized struct autofs_dev_ioctl with the path
  264. field set to the mount point in question and the size field adjusted
  265. appropriately. Upon return the uid field of struct args_requester contains
  266. the uid and gid field the gid.
  267. When reconstructing an autofs mount tree with active mounts we need to
  268. re-connect to mounts that may have used the original process uid and
  269. gid (or string variations of them) for mount lookups within the map entry.
  270. This call provides the ability to obtain this uid and gid so they may be
  271. used by user space for the mount map lookups.
  272. AUTOFS_DEV_IOCTL_EXPIRE_CMD
  273. ---------------------------
  274. Issue an expire request to the kernel for an autofs mount. Typically
  275. this ioctl is called until no further expire candidates are found.
  276. The call requires an initialized struct autofs_dev_ioctl with the
  277. ioctlfd field set to the descriptor obtained from the open call. In
  278. addition an immediate expire, independent of the mount timeout, can be
  279. requested by setting the how field of struct args_expire to 1. If no
  280. expire candidates can be found the ioctl returns -1 with errno set to
  281. EAGAIN.
  282. This call causes the kernel module to check the mount corresponding
  283. to the given ioctlfd for mounts that can be expired, issues an expire
  284. request back to the daemon and waits for completion.
  285. AUTOFS_DEV_IOCTL_ASKUMOUNT_CMD
  286. ------------------------------
  287. Checks if an autofs mount point is in use.
  288. The call requires an initialized struct autofs_dev_ioctl with the
  289. ioctlfd field set to the descriptor obtained from the open call and
  290. it returns the result in the may_umount field of struct args_askumount,
  291. 1 for busy and 0 otherwise.
  292. AUTOFS_DEV_IOCTL_ISMOUNTPOINT_CMD
  293. ---------------------------------
  294. Check if the given path is a mountpoint.
  295. The call requires an initialized struct autofs_dev_ioctl. There are two
  296. possible variations. Both use the path field set to the path of the mount
  297. point to check and the size field adjusted appropriately. One uses the
  298. ioctlfd field to identify a specific mount point to check while the other
  299. variation uses the path and optionally in.type field of struct args_ismountpoint
  300. set to an autofs mount type. The call returns 1 if this is a mount point
  301. and sets out.devid field to the device number of the mount and out.magic
  302. field to the relevant super block magic number (described below) or 0 if
  303. it isn't a mountpoint. In both cases the the device number (as returned
  304. by new_encode_dev()) is returned in out.devid field.
  305. If supplied with a file descriptor we're looking for a specific mount,
  306. not necessarily at the top of the mounted stack. In this case the path
  307. the descriptor corresponds to is considered a mountpoint if it is itself
  308. a mountpoint or contains a mount, such as a multi-mount without a root
  309. mount. In this case we return 1 if the descriptor corresponds to a mount
  310. point and and also returns the super magic of the covering mount if there
  311. is one or 0 if it isn't a mountpoint.
  312. If a path is supplied (and the ioctlfd field is set to -1) then the path
  313. is looked up and is checked to see if it is the root of a mount. If a
  314. type is also given we are looking for a particular autofs mount and if
  315. a match isn't found a fail is returned. If the the located path is the
  316. root of a mount 1 is returned along with the super magic of the mount
  317. or 0 otherwise.