܍~c UdZddlZddlZddlZddlmZddlmZddlmZm Z ddl m Z ddl m Z ddlmZmZdd lmZdd lmZejd Zejd Zejd ZejdZejdZejdZejdZddiZejeZ dZ!ddde!egeedgddgdZ"ee#d<ee"Zde$de de d ed!e%d"df d#Z&d$Z'd%Z(dBd&Z)dCd(Z*d)Z+dCd*Z,d+Z-d,Z. dDd.Z/d/Z0d0Z1d1Z2d2Z3d3Z4d4Z5d5Z6d6Z7d7Z8d8Z9d9Z:d:Z;d;Zd>Z?d?Z@d@ZAdAZBdS)Ez1Disk Setup: Configure partitions and filesystems.N)Logger)dedent)subputil)Cloud)Config) MetaSchema get_meta_doc) ALL_DISTROS) PER_INSTANCEsfdisksgdisklsblkblkidblockdev partprobewipefsLANGCaiThis module is able to configure simple partition tables and filesystems. .. note:: for more detail about configuration options for disk setup, see the disk setup example .. note:: if a swap partition is being created via ``disk_setup`` then a ``fs_entry`` entry is also needed in order for mkswap to be run, otherwise when swap activation is later attempted it will fail. For convenience, aliases can be specified for disks using the ``device_aliases`` config key, which takes a dictionary of alias: path mappings. There are automatic aliases for ``swap`` and ``ephemeral``, where ``swap`` will always refer to the active swap partition and ``ephemeral`` will refer to the block device of the ephemeral image. Disk partitioning is done using the ``disk_setup`` directive. This config directive accepts a dictionary where each key is either a path to a block device or an alias specified in ``device_aliases``, and each value is the configuration options for the device. File system configuration is done using the ``fs_setup`` directive. This config directive accepts a list of filesystem configs. cc_disk_setupz Disk Setupz$Configure partitions and filesystemsa| device_aliases: my_alias: /dev/sdb swap_disk: /dev/sdc disk_setup: my_alias: table_type: gpt layout: [50, 50] overwrite: true swap_disk: table_type: gpt layout: [[100, 82]] overwrite: true fs_setup: - label: fs1 filesystem: ext4 device: my_alias.1 cmd: mkfs -t %(filesystem)s -L %(label)s %(device)s - label: fs2 device: my_alias.2 filesystem: ext4 - label: swap device: swap_disk.1 filesystem: swap mounts: - ["my_alias.1", "/mnt1"] - ["my_alias.2", "/mnt2"] - ["swap_disk.1", "none", "swap", "sw", "0", "0"] disk_setupfs_setup)idnametitle descriptiondistros frequencyexamplesactivate_by_schema_keysmetarcfgcloudlogargsreturnc |di  fd}|d}t|trt|||dt ||D]\}}t|ts|d|z3 |dtj tjd|zt||fv#t$r'} tj td | zYd } ~ d } ~ wwxYw|d } t| tr|d t | t| || D]}t|ts|d |z0 |d|d} tj tjd| zt |f#t$r'} tj td| zYd } ~ d } ~ wwxYwd Sd S)z[ See doc/examples/cloud-config-disk-setup.txt for documentation on the format. device_aliasesc`|}|p|p|SN)getdevice_name_to_device)candrr#r(s @/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.pyalias_to_devicezhandle..alias_to_devicets3!!$''**4<488@D@rzPartitioning disks: %szInvalid disk definition for %sz!Creating new partition table/diskzCreating partition on %s)logfuncmsgfuncr%z Failed partitioning operation %sNrzsetting up filesystems: %sz"Invalid file system definition: %szCreating new filesystem.devicezCreating fs for %sz%Failed during filesystem operation %s)r+ isinstancedictupdate_disk_setup_devicesdebugstritemswarningrlog_timeLOGmkpart Exceptionlogexclistupdate_fs_setup_devicesmkfs) rr"r#r$r%r/rdisk definitionerr4r(s ` @r.handlerGksWW-r22NAAAAAA&&J*d##J!*o>>> *C OO<<< * 0 0 2 2 J J D*j$//  >> I2T9 +   J J J C!Dq!HIIIIIIII Jwwz""H(D!!O .H >>>/:::" O OJj$//  @:MNNN O 4555#11 I,v5$   O O O C!IA!MNNNNNNNN O#OO O Os2AD D7D22D7 AH  I*I  Ic8t|D]}||}|||kr||vrtd||||=||||<t||tr |||d<||=td||dS)Nz0Replacing %s in disk_setup for translation of %s _orignamez,updated disk_setup device entry '%s' to '%s')rAr=infor5r6r8)rtformerorigname transformeds r.r7r7s$$  gh''  +"9"9  * $ $ HHB    ;'",X"6 ; j-t 4 4 <3;J{ #K 0 x  :      !  r0c|D]}t|tstd|3|d}|Kt j|\}}||}|)|}td|||||d<||d<|r1d|vr(td||||d|d<||d<dS)Nz"entry in disk_setup not a dict: %sr4z%s is mapped to disk=%s part=%srI partitionzKPartition '%s' from dotted device name '%s' overrides 'partition' key in %s _partition)r5r6r=r;r+rexpand_dotted_devnamer8)rrKrErLdevparttformeds r.rBrBs! + + *d++  KK>(++   0:: d'#,,  C II18Wd   '/J{ ##*Jx  +j(( 6 ,6k+B <(&*J{ #A + +r0c#zKtj|}|r ||d}d|DD] \}}||fV dS)zd Returns the key/value pairs of output sent as string like: FOO='BAR' HOME='127.0.0.1' Nc8g|]}|dS)=)split.0xs r. z"value_splitter..s"555qwws||555r0)shlexrX)valuesstart_valueskeyvalues r.value_splitterrcsj k&!!G "%&&/55W555 U5jr0Fc#Ktddd|g}|r|dd} tj|\}}n(#t$r}td|d||d}~wwxYwd|D}|D]9}ddddd }t |D]\} } | || <|V:dS) a Enumerate the elements of a child device. Parameters: device: the kernel device name nodeps : don't enumerate children devices Return a dict describing the disk: type: the entry type, i.e disk or part fstype: the filesystem type, if it exists label: file system label, if it exists name: the device name, i.e. sda z--pairsz--outputzNAME,TYPE,FSTYPE,LABELz--nodepsNFailed during disk check for  c\g|])}t|dk'|*S)r)lenrXrYs r.r\z"enumerate_disk..s0 J J J1s17799~~7I7IQ7I7I7Ir0)rtypefstypelabel) LSBLK_CMDappendrr?strip splitlinesrclower) r4nodeps lsblk_cmdrJ_errrFpartsrSdrarbs r.enumerate_diskrvs5   I%$$$ DYy)) dd i5;VVQQ ?    K J1133 J J JE     ).. # #JC"AciikkNN  sA A& A!!A&cnt|dD]"}d|vr|dcS#dS)z@ Return the device type of the device by calling lsblk. T)rqriN)rvrp)r4rus r. device_typerx!sM F4 0 0 0%% Q;;V9??$$ $ $ $  4r0cd} t|}n,#t$rtd|YdSwxYw|r|dkrdS|s|dkrdSdS)z0 Check if the device is a valid device. zQuery against device %s failedFrSTrD)rxr?r=r;)rrOd_types r.is_device_validr|,sFT""  4d;;;uuVv%%t 6V++t 5s %==cd\}}}}tdd|g} tj|ddg\}}n(#t$r}td|d||d }~wwxYw|rt|d krgt |d D]U\}} |d kr| } |d kr| };|dkr| }V|||fS)z Check if the device has a filesystem on it Output of blkid is generally something like: /dev/sda: LABEL="Backup500G" UUID="..." TYPE="ext4" Return values are device, label, type, uuid )NNNNz-cz /dev/nullr)rcsrerfN)r_rkriuuid) BLKID_CMDrr?rhrorcrp) r4outrkfs_typer blkid_cmdrsrFrarbs r.check_fsr>s(!7CD+v6IIiaV444 TT i5;VVQQ ?     ! s~~ A % %,S::: ! ! U99;;'))!EEYY[[F**#GGYY[[F** D '4 s0 AAAc,t|\}}}|S)z7 Returns true if the device has a file system. )r)r4_rs r. is_filesystemr^sV$$MAw Nr0Tc|d}|sddg}d}t|D]}|d|kr|durd|dzdfcS|d|kr|r |d |ks|sd|dzd fcS|d |vr:|d dks|drd }|d dkry|dsd|dzdfcS|s|dfStd d S)a Find a device that is either matches the spec, or the first The return is value is (, ) where the device is the device to use and the bool is whether the device matches the fs_type and label. Note: This works with GPT partition tables! NrzrDrSFrj/dev/%srrkTriz5Failed to find device during available device search.)NF)rvr=r;)r4rrk valid_targets label_match replace_fsraw_device_usedrus r.find_device_noderfsE$ } )( O F # #66 X;* $ $)=)=& )51 1 1 1 X;' ! !  "wZ5000& )40 0 0 0 V9 % %yF""ak""&yF""x[ 6!AfI-u5555 KKGHHH =r0cttt|dkrdSt|\}}}|rdSdS)z Check if the device is currently used. Returns true if the device has either a file system or a partition entry is no filesystem found on the disk. rTF)rhrArvr)r4r check_fstypes r. is_disk_usedrsT 4v&& ' '((1,,t"&))A|Qt 5r0c"t|dkrtd|d|dz}|dd} |rt||St|S#t$r}td|z|d}~wwxYw)a- Call the appropriate function. The first value is the template for function name The second value is the template replacement The remain values are passed to the function For example: get_dyn_func("foo_%s", 'bar', 1, 2, 3,) would call "foo_bar" with args of 1, 2, 3 r~z*Unable to determine dynamic funcation namerrNzNo such function %s to call!)rhr?globalsKeyError)r% func_name func_argsrFs r. get_dyn_funcrs 4yy1}}DEEEQ$q'!IQRRIK  ('799Y'3 399Y' ' KKK6BCCJKsA,A,, B6B  Bc tjtd|g\}}tjtd|g\}}n(#t$r}td|d||d}~wwxYwt|t|z S)Nz --getsize64z--getsszFailed to get z size )r BLKDEV_CMDr?int)r4 size_in_bytesr sector_sizerFs r. get_hdd_sizersJ9j-%HII qJ 6#BCC QQ JJJivvvqqABBIJ }  K 0 0 00s>A A& A!!A&c t|td|g} tj|d|z\}}n(#t$r}td|d||d}~wwxYwg}|D]}|}t |dkr*||dvr|dd vrQd} ttd t |d D]2} ||  r|| d kr || } n3| | |S)z Returns true if the partition layout matches the one on the disk Layout should be a list of values. At this time, this only verifies that the number of partitions and their labels is correct. z-l%s data#Error running partition command on rfNr)extendedemptyrT)reverse/) read_parttbl SFDISK_CMDrr?rorXrhrpsortedrangeisdigitrm) r4layoutprt_cmdrrsrF found_layoutline_line type_labelr[s r.check_partition_mbr_layoutrsq4(GIgFVO<<< TT i;A6611 E    L  ,,  u::??  U1X  Ry  $999JE!SZZ00$???  8##%%%(c//!&qJE    + + + s8 AAActd|g} tj|t\}}n(#t$r}td|d||d}~wwxYwt |}|D]+}|drn,d|D}g} |D]I} t| dkr| dr | d d } | | J| S) Nz-p) update_envrrfNumbercfg|].}|d/S))rnrX)rZrs r.r\z.check_partition_gpt_layout..s1 ; ; ;TZZ\\   ! !! $ ; ; ;r000rr~) SGDISK_CMDr LANG_C_ENVr?iterrorn startswithrhendswithrm) r4rrrrsrF out_linesrcodescleanedcodes r.check_partition_gpt_layoutrs44(GIg*=== TT i;A6611 E    S^^%%&&I" ::<< " "8 , ,  E  < ; ; ; ;EG t99>>dmmD11>!9Dt Ns+ AA  Actd|||}td||||t|tr|rt |dkrdSdSt |t |kred|D}td||t ||D]*\}}|#t|t|krdS+dSdS)z See if the partition lay out matches. This is future a future proofing function. In order to add support for other disk layout schemes, add a function called check_partition_%s_layout zcheck_partition_%s_layoutz6called check_partition_%s_layout(%s, %s), returned: %srTFctg|]5}t|ttfrt|dnd6S)rN)r5tuplerAr9rYs r.r\z*check_partition_layout..DsF   DEAt}55 ?C!III4   r0zLayout types=%s. Found types=%s)rr=r8r5boolrhzipr9) table_typer4rr layout_typesitypeftypes r.check_partition_layoutr's  #ZLII@ &$  c,''1,,4u \  c&kk ) )  IO    -|\    l;;  LE5 SZZ3u::%=%=uut 5r0c&t|tst|trdSt|dkrt|tst|tst dt|}|dkrt dg}d}|D]}d}|}|dz }t|tr*t|dkrt d |z|\}}t t |t |d z z}||kr|d |z|d |d |d |} t|dkrt d| z| S)a@ Calculate the layout of the partition table. Partition sizes are defined as percentage values or a tuple of percentage and partition type. For example: [ 33, [66: 82] ] Defines the first partition to be a size of 1/3 the disk, while the remaining 2/3's will be of type Linux Swap. z0,rzPartition layout is invalidrz$Only simply partitioning is allowed.Srr~%Partition was incorrectly defined: %sdz,,%s,rfz-Calculated partition definition is too big %s) r5rArrhr?rfloatrmjoin) sizer last_part_numpart_definitionpart_numrS part_typepercent part_sizesfdisk_definitions r.get_partition_mbr_layoutrRs fd # # 64(@(@t F qZ55j??5666KKMq>???OHFF A  dD ! ! &4yyA~~ G$ NOOO!% GYd uW~~';<== } $ $  " "6I#5 6 6 6 6  " " "yyy))#D E E E E /22 ?a <    r0ct|trdddgfgSg}|D]}t|tr+t|dkrt d|z|\}}n|}d}t t |t |dz z}||dd|gfd|ddd<|S)Nrr~rrz+{}r) r5rrArhr?rrrmformat)rrpartition_specsrOrpartition_typers r.get_partition_gpt_layoutrs&$ 1vO O O i & & "9~~"";iG'0 #G^^G!Nd uW~~';<== ELL4K4K0LMNNNN#$OBB r0cJd}d}d}t|d5}|||z|| tj|||z|dddn #1swxYwYt |dS)Nizrb+)openwriteseekosSEEK_ENDflushr)r4null start_lenend_lenfps r.purge_disk_ptablers DIG fe   #$$$ "+&&&       sA&B  B B cFt|D]}|ddvrutdd|dzg} td|dt j|V#t $r}t d|dz|d}~wwxYwt |dS) z' Remove parition table entries ri)rDcryptz--allrrzPurging filesystem on /dev/%szFailed FS purge of /dev/%sN)rv WIPEFS_CMDr=rJrr?r)r4ru wipefs_cmdrFs r. purge_diskrs F # #   V9- - -$gy1V9/DEJ 8!F)DDD *%%%%   01V9<  .fs5A&& B0B  Bc&td|||S)z Call the appropriate function for creating the table definition. Returns the table definition This is a future proofing function. To add support for other layouts, simply add a "get_partition_%s_layout" function. zget_partition_%s_layoutr)rrrs r.get_partition_layoutrs 1:tV L LLr0ct t|g}n td|g}tj t j|n4#t $r'}tjtd|zYd}~nd}~wwxYwtjdS)zq `Partprobe` is preferred over `blkdev` since it is more reliably able to probe the partition table. Nz --rereadptz%Failed reading the partition table %s) PARTPROBE_CMDrrudevadm_settlerr?r@r=)r4 probe_cmdrFs r.rrs  "F+ v6 F ) FFF C@1DEEEEEEEEF sA A6A11A6ctddd|g} tj|d|zn(#t$r}td|d||d}~wwxYwt|dS) zV Break out of mbr partition to allow for future partition types, i.e. gpt z--Linuxz--unit=Sz--forcerrzFailed to partition device rfN)rrr?r)r4rrrFs r.exec_mkpart_mbrrs 9j)VDG '00000 i396611 =    s( A AA c  tjtd|gt|D]\}\}\}}|dz }tjtdd||||g|Tt |dd}tjtdd|||gn*#t $rtd |wxYwt|dS) Nz-Zrz-nz{}:{}:{}r0z-tz{}:{}zFailed to partition device %s) rr enumeraterr9ljustr?r=r;r)r4rindexrr_endpinputs r.exec_mkpart_gptrs) :tV,---5>v5F5F   1E1NLUC QJE I%%eUC88    )^,,221c:: w~~eV'D'DfM "  3V<<< s CC'C.c&td|||S)a Fetches the function for creating the table type. This allows to dynamically find which function to call. Paramaters: table_type: type of partition table to use device: the device to work on layout: layout definition specific to partition table zexec_mkpart_%sr)rr4rs r. exec_mkpartr s (*ff E EEr0ctj|sDtjtj|st d|ztjdS)z?Assert that device exists and settle so it is fully recognized.zBDevice %s did not exist and was not created with a udevadm settle.N)rpathexistsrr RuntimeErrorr4s r.assert_and_settle_devicerss 7>>& ! ! w~~f%% )+12  r0ct|tj|}td||dd}|dd}|dd}tdt|tr|r|stdd Std |t|s#td | t|trC| d kr+tdt|d Stdt|||rtddStd|s;t!|st#|rtd|d Std|t%|}tdt'|||}td|td|t)|||td|d S)a Creates the partition table. Parameters: definition: dictionary describing how to create the partition. The following are supported values in the dict: overwrite: Should the partition table be created regardless of any pre-exisiting data? layout: the layout of the partition table table_type: Which partition table to use, defaults to MBR device: the device to work on. z!Checking values for %s definition overwriteFrrmbrz Checking against default devicesz)Device is not to be partitioned, skippingNz'Checking if device %s is a valid devicez%Device {device} is not a disk device!rremovez,Instructed to remove partition table entriesz!Checking if device layout matchesz"Device partitioning layout matchesTz'Checking if device is safe to partitionz-Skipping partitioning on configured device %szChecking for device size of %szCalculating partition layoutz Layout is: %szCreating partition table on %szPartition table created for %s)rrr realpathr=r8r+r5rr|r?rr9rprrrrrrr )r4rErrr device_sizers r.r>r>+sV$$$ W  f % %FII16:::{E22I ^^He , ,F e44JII011164   =>>>II7@@@ 6 " "  3 : :& : I I   &#6<<>>X#=#= @AAA6II1222j&&99 6777tII7888 ,v..-2G2G A6JJJII.777v&&KII,---*:{FKKOII /222II.777 FO444II.77777r0cdddddd}d|vrd}||vr||Std|dS)z9 A force flag might be -F or -F, this look it up z-Fz-f)extbtrfsxfsreiserfsswaprzForce flag for %s is unknown.rz)rpr=r;)fsflagss r.lookup_force_flagrmsp    E    xxzzURyKK/444 2r0c |d}|d}t|dd}|d}|dg}|dg}|dd }|d d }t|tj|}t d ||r|r|r"||}t d ||t d |t|\} } } t d|| | | |kr]| |krWt d||st d|dSt d|nlt d|nO|rt| dvr|} t d|d} | dkrd } t|||| |\}}t d| ||rt ddS|s|r|rt d||st d|dSn_|r%t| dkrt d|nt ddSt d||||st d |dS|s%|s#td!|"d }|rL|d|||d#z}d}|rt d$||rt d%|ntjd&|z}|stjd'|z}|st d(||dS|g}|r|d)|g|st%|d*kr&t'|}|r|||r||||t d+||t d,t| tj||-dS#t$r}td.|d/||d}~wwxYw)0a Create a file system on the device. label: defines the label to use on the device fs_cfg: defines how the filesystem is to look The following values are required generally: device: which device or cloud defined default_device filesystem: which file system type overwrite: indiscriminately create the file system partition: when device does not define a partition, setting this to a number will mean device + partition. When set to 'auto', the first free device or the first device which matches both label and type will be used. 'any' means the first filesystem that matches on the device. When 'cmd' is provided then no other parameter is required. rkr4rOany filesystemcmd extra_optsrFrz#Checking %s against default devicesz%Manual request of partition %s for %szChecking device %sz0Device '%s' has check_label='%s' check_fstype=%sz Existing file system found at %sz"Device %s has required file systemNzDestroying filesystem on %sz"Device %s is cleared for formating)autor!z,Identifying device to create %s filesytem onT)rrkrrz(Automatic device for %s identified as %sz+Found filesystem match, skipping formating.z*Replacing file system on %s as instructed.zENo device available that matches request. Skipping fs creation for %snonez.Using the raw device to place filesystem %s onz(Error in device identification handling.z;File system type '%s' with label '%s' will be created on %szDevice is not known: %szENo way to create filesystem '{label}'. fs_type or fs_cmd must be set.)rk)rkr"r4z8fs_setup:overwrite ignored because cmd was specified: %sz9fs_setup:extra_opts ignored because cmd was specified: %szmkfs.%szmk%sz.Cannot create fstype '%s'. No mkfs.%s commandz-LrDzCreating file system %s on %sz Using cmd: %s)shellzFailed to exec of 'z': )r+r9rrr rr=r8rrr;rprr?rrwhichextendrxrrm)fs_cfgrkr4rOrfs_cmdfs_opts fs_replacer check_labelrrodevicerreuser'mkfs_cmd force_flagrFs r.rCrCs* JJw  E ZZ ! !FFJJ{E2233Ijj&&G ZZr " "Fjjr**GL%00J ;..IV$$$ W  f % %FII3V<<< D ))++D      %vyy1F II7F    &///'/'7'7$ \1 >        %  LG$;$; II8& A A A C >GGG 96BBBB II:F C C C C &s9~~++--@@ @%HHH ??   % %K( #!     rrCr0r.rAs 877  !!!!!!######<<<<<<<<))))))++++++TZ ! ! TZ ! ! DJw   DJw   TZ # #  ;'' TZ ! ! c] g!!6   3%}    B!-j9Q))j)))V ,t  4O 4O4O#(4O/54O=A4O 4O4O4O4On   4#+#+#+L    ////d$   @   5555p(KKK6111$$$N&&&R(((V444n0   ( M M M$"6 F F F    ?8?8?8D,wLwLwLwLwLr0