ph; hUdZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Zddl m Z m Z mZddlmZdd lmZdd lmZmZdd lmZejead Zd ZdZdZeZddgZ dddede edgegdZ!ee"d<ee!ZdZ#dZ$dZ%dZ&dddZ'd d dZ(d!d"gZ)gd#Z*dMd$Z+d%e,d&ed'ed(ed)e-d*df d+Z.d,Z/d-Z0dNd.Z1dNd/Z2dNd0Z3d1Z4d2Z5d3Z6d4Z7d5Z8d6Z9d7Z:d8Z;d9Zd=Z? dQd>Z@d?ZAd@ZBdAZCdBZDdCZEdDZFdEZGdFZHdGZIdNdHZJdIZK dRdKZLdLe4iZMdS)Sz*Apt Configure: Configure apt for the user.N)Logger)dedent)gpg)log)subp templaterutil)Cloud)Config) MetaSchema get_meta_doc) PER_INSTANCEz ^[\w-]+:\wz/etc/apt/trusted.gpgz/etc/apt/trusted.gpg.d/z/etc/apt/cloud-init.gpg.d/ubuntudebiancc_apt_configurez Apt ConfigurezConfigure apt for the usera This module handles both configuration of apt options and adding source lists. There are configuration options such as ``apt_get_wrapper`` and ``apt_get_command`` that control how cloud-init invokes apt-get. These configuration options are handled on a per-distro basis, so consult documentation for cloud-init's distro support for instructions on using these config options. .. note:: To ensure that apt configuration is valid yaml, any strings containing special characters, especially ``:`` should be quoted. .. note:: For more information about apt configuration, see the ``Additional apt configuration`` example.a- apt: preserve_sources_list: false disable_suites: - $RELEASE-updates - backports - $RELEASE - mysuite primary: - arches: - amd64 - i386 - default uri: 'http://us.archive.ubuntu.com/ubuntu' search: - 'http://cool.but-sometimes-unreachable.com/ubuntu' - 'http://us.archive.ubuntu.com/ubuntu' search_dns: false - arches: - s390x - arm64 uri: 'http://archive-to-use-for-arm64.example.com/ubuntu' security: - arches: - default search_dns: true sources_list: | deb $MIRROR $RELEASE main restricted deb-src $MIRROR $RELEASE main restricted deb $PRIMARY $RELEASE universe restricted deb $SECURITY $RELEASE-security multiverse debconf_selections: set1: the-package the-package/some-flag boolean true conf: | APT { Get { Assume-Yes 'true'; Fix-Broken 'true'; } } proxy: 'http://[[user][:pass]@]host[:port]/' http_proxy: 'http://[[user][:pass]@]host[:port]/' ftp_proxy: 'ftp://[[user][:pass]@]host[:port]/' https_proxy: 'https://[[user][:pass]@]host[:port]/' sources: source1: keyid: 'keyid' keyserver: 'keyserverurl' source: 'deb [signed-by=$KEY_FILE] http:/// bionic main' source2: source: 'ppa:' source3: source: 'deb $MIRROR $RELEASE multiverse' key: | ------BEGIN PGP PUBLIC KEY BLOCK------- ------END PGP PUBLIC KEY BLOCK------- source4: source: 'deb $MIRROR $RELEASE multiverse' append: false key: | ------BEGIN PGP PUBLIC KEY BLOCK------- ------END PGP PUBLIC KEY BLOCK-------)idnametitle descriptiondistrosexamples frequencyactivate_by_schema_keysmetaz/var/lib/apt/listsz'/etc/apt/apt.conf.d/94cloud-init-configz)/etc/apt/apt.conf.d/90cloud-init-aptproxyzkeyserver.ubuntu.comz!http://archive.ubuntu.com/ubuntu/z"http://security.ubuntu.com/ubuntu/PRIMARYSECURITYz$http://ports.ubuntu.com/ubuntu-portsamd64i386)s390xarm64armhfpowerpcppc64elriscv64c|tj|}|tvrtS|t vrt Std|z)zreturns the default mirrors for the target. These depend on the architecture, for more see: https://wiki.ubuntu.com/UbuntuDevelopment/PackageArchive#PortsNz#No default mirror known for arch %s)r get_dpkg_architecturePRIMARY_ARCHESPRIMARY_ARCH_MIRRORScopy PORTS_ARCHES PORTS_MIRRORS ValueError)archtargets C/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.pyget_default_mirrorsr1sg |)&11 ~#((*** |!!### :TA B BBrcfgcloudrargsreturnc(d}||at|}|di}t|ts0t dt|t||t|||dS)zprocess the config for apt_config. This can be called from curthooks if a global apt config was provided or via the "apt" standalone command.Naptz9Expected dictionary for 'apt' config, found {config_type}) config_type) LOGconvert_to_v3_apt_formatget isinstancedictr-formattypeapply_debconf_selections apply_apt)rr3r4rr5r/apt_cfgs r0handlerDsF  "3 ' 'CggeR  G gt $ $  G N N MM O     Wf--- guf%%%%%r2ctjrdStjdstjdsdSdS)N)Fzsystem is snappy.zapt-getr8)Fzno apt commands.)TzApt is available.)r system_is_snappyrwhichr2r0_should_configure_on_empty_aptrIsH *)) Jy ! !)TZ%6%6)(( $ $r2c|s0t\}}|std|dStd|tj|d}tj|}t |||}td|tj|ddrt|||n[tj |d dr3t||t||||t||| t|tt n1#t"t$f$rtd YnwxYwd |vrf|}||d <|d |d <d} |dt(} | rt+j| j} t1|d |||| dSdS)Nz#Nothing to do: No apt config and %szhandling apt config: %sr/codename)r.zApt Mirror info: %sgenerate_mirrorlistsFpreserve_sources_listz)Failed to apply proxy or apt config info:sourcesRELEASEMIRRORadd_apt_repo_match)r/template_params aa_repo_match)rIr:debugr lsb_releaser'find_apt_mirror_infois_truer<rMis_falseadd_mirror_keysgenerate_sources_listrename_apt_listsapply_apt_config APT_PROXY_FN APT_CONFIG_FNIOErrorOSError exceptionADD_APT_REPO_MATCHrecompilesearchadd_apt_sources) r3r4r/ should_configmsgreleaser.mirrorsparamsmatchermatchcfgs r0rBrBs" ;== s  II;S A A A FII'---f---j9G  %f - -D"3D999GII#W--- |CGG2E::;;0S'51111 sww6>> ? ?0V$$$c7GU;;;&$///ClM:::: W CCC ABBBBBCC#y"8,x77/1CDD  2j**1G  N "!       sE+F  F cl|ds|dz }tjdg||ddS)N zdebconf-set-selectionsTdatar/capture)endswithr) selectionsr/s r0debconf_set_selectionsrvsT   u % %e I !"  r2cg}g}|D]f}|tvrFtd|t||||Q||gt |rtd|t |r,t jddgt|zd|ddSdS)Nzunconfiguring %szSThe following packages were installed and preseeded, but cannot be unconfigured: %szdpkg-reconfigurez--frontend=noninteractiveTrq)CONFIG_CLEANERSr:rUappendlenwarningrlist)packagesr/ unhandled to_configpkgs r0dpkg_reconfigurer s II"" / ! ! II(# . . . C  ( ( (   S ! ! ! !   S ! ! ! ! 9~~   -     9~~  !< =9oo          r2c< |d stddSd fdt D}t ||t} D]]\}}| D]C}| drtj dd |}||D^tj|}td |||} t%| d krtd dSt'| |dS) z2apply_debconf_selections - push content to debconfdebconf_selectionsz(debconf_selections was not set in configN c g|] }| SrHrH).0keyselsetss r0 z,apply_debconf_selections..7sKKKSGCLKKKr2rK#z[:\s].*z pkgs_cfgd: %srzno need for reconfig)r<r:rUjoinsortedkeysrvencodesetitems splitlines startswithrdsubaddr get_installed_packages intersectionrzr) r3r/ru pkgs_cfgd_keycontentlinerpkgs_installed need_reconfigrs @r0rArA,s gg*++G  <===KKKKF7<<>>4J4JKKKLLJ:,,..v>>>>I  g&&((  Ds## &R..C MM#       088NIIoy)))**>::M =Q ()))]6222222r2ctjtj|d}td||D]}t j|dS)z%clean out any local cloud-init configz/etc/cloud/cloud.cfg.d/*dpkg*z#cleaning cloud-init config from: %sN)globr target_pathr:rUosunlink)r/flistdpkg_cfgs r0clean_cloud_initrOsg I !@AA  EII3U;;; (r2c|}|dr |dd}|d}|dkr ||dzd}|dd}|S)zmirrorurl_to_apt_fileprefix Convert a mirror url to the file prefix used by apt on disk to store cache information for that mirror. To do so do: - take off ???:// - drop tailing / - convert in string / to _/rz://N_)rtfindreplace)mirrorstringposs r0mirrorurl_to_apt_fileprefixrZsnF s" ++e  C axxa " ^^C % %F Mr2ct|}tj|t}|D]\}}||}|s|t jjzt|z}|t jjzt|z} || krmt|} tj d|zD]n} | | | d} t d| |  t j| | C#t$rtddYkwxYwdS)z>rename_apt_lists - rename apt lists to preserve old cache dataz%s_*NzRenaming apt list %s to %szFailed to rename apt list:T)exc_info)r1rr APT_LISTSrr<rpathseprrzrr:rUrenamerar{) new_mirrorsr/r.default_mirrorspreromirrornmirroroprefixnprefixolenfilenamenewnames r0r\r\lse)$//O  69 - -C*0022IIw//$''   #&A'&J&JJ #&A'&J&JJ g   7|| &7"233 I IH '$%%9G II2Hg F F F I (G,,,, I I I 84 HHHHH I  IIIs>D&D=<D=cn||vrtd|||||S)zmirror_to_placeholder replace the specified mirror in a template with a placeholder string Checks for existance of the expected mirror and warns if not foundz%Expected mirror '%s' not found in: %s)r:r{r)tmplr placeholders r0mirror_to_placeholderrs:T ;VTJJJ << , ,,r2cNdddddd} ||}n#t$r|}YnwxYw|S)zthere are a few default names which will be auto-extended. This comes at the inability to use those names literally as suites, but on the other hand increases readability of the cfg quite a lotz$RELEASE-updatesz$RELEASE-backportsz$RELEASE-securityz$RELEASE-proposedz$RELEASE)updates backportssecurityproposedrj)KeyError)suitemappingretsuites r0map_known_suitesrsX &)'' G5>  Os  ""c|s|S|}|D]}t|}tj|d|i}td||d}|dD]}|dr||z }|}t|dkrWd} |ddr)|dd D]} | dz } | d rn|| |krd |z}||z }|}|S) zRreads the config for suites to be disabled and removes those from the templaterPzDisabling suite %s as %srTr[N]z"# suite disabled by cloud-init: %s) rr render_stringr:rUrrsplitrzrt) disabledsrcrjretsrcr releasesuitenewsrcrcolspcolcols r0disable_suitesrsO  F '' .uy'6JKK  ,e\BBB%%d++  Ds## $ ::<4 ( (D | DEEE11 !2 3   F55nEEK  KKH    F~k**&tV44Hcgg&6777KKHOFH5111111r2cdtjd}tjt |tjt ||jjdz |dddtjt ||jjdz |dddd S) zUgenerate_mirrorlists create one file for every mirror for apt-transport-mirror(1)z/etc/apt/mirrors.listrrrrz-security.listrN)pathlibPathr ensure_dirstrrrr)r3rkr4aptmirs r0rMrMs\, - -FOCKK   O F )000 011 9 !!!   O F )999 9:: : """ r2Fctd| tj|j}t d|||S#t j$rtdwxYw)zM actual adding of a key as defined in key argument to the system zAdding key: '%s'r) output_filerrhardenedz(failed to add apt GPG Key to apt keyring) r:rUrrstemapt_keyrProcessExecutionErrorrb)rrrr/rs r0add_apt_key_rawrs{ II!3'''|I&&+u$S8LLLL  % @AAA s +A +A4cd|vr5d|vr1t}d|vr|d}tj|d||d<d|vr t|d|p|d|SdS)z Add key to the system as defined in ent (if any). Supports raw keys or keyid's The latter will as a first step fetched to get the raw key keyidr keyserverrrN)DEFAULT_KEYSERVERr getkeybyidr)entr/rrrs r0rr s #~~%s**% #  K(I^CL)<<E  || J 4S_x    |r2c8|jdSN)rupdate_package_sources)r4s r0update_packagesr s L'')))))r2c*|i}|tdt|tstd|z|D]}||}td|d|vr||d<d|vr"d|dvrt ||d }||d <nt ||}d|vrj|d}tj||}|d d s)tj d |d|d<|d d s|dxxd z cc<||rK tjdd|g|n.#tj$rtdwxYwLtj||d} d|z} d} d|vr |dsd} t%j| | | #t($r"} td| | d} ~ wwxYwt+|dS)a install keys and repo source .list files defined in 'sources' for each 'source' entry in the config: 1. expand template variables and write source .list file in /etc/apt/sources.list.d/ 2. install defined keys 3. update packages via distro-specific method (i.e. apt-key update) @param srcdict: a dict containing elements required @param cloud: cloud instance object Example srcdict value: { 'rio-grande-repo': { 'source': 'deb [signed-by=$KEY_FILE] $MIRROR $RELEASE main', 'keyid': 'B59D 5F15 97A5 04B7 E230 6DCA 0620 BBCF 0368 3F77', 'keyserver': 'pgp.mit.edu' } } Note: Deb822 format is not supported Nz did not get a valid repo matcherzunknown apt format: %szadding source/key '%s'rsourcez $KEY_FILETrKEY_FILErz/etc/apt/sources.list.d/rzadd-apt-repositoryz --no-updaterKzadd-apt-repository failed.z%s aryw)omodezfailed write to file %s: %s)r-r=r> TypeErrorr:rUrrrrrrrrtrrrbrr rr`r ) srcdictr4r/rSrTrrkey_filer sourcefncontentsrdetails r0rgrg!s6;<<< gt $ $>0G<===..h *C000 S &C O s??{c(m;;"3>>>H*2OJ ' '"3//H 3   X(AA:))#..  gll*C OC O:''00 '  OOOw &OOO =   )=&A!-    :;;;  #FC O<< (HE3s8} OHhe < < < < <    MM76 J J J  E Fs$E+F',G HG<<HcNi}tdt|trLtd|D].}d|vrd|d<t j|d}n|d}|||</n't|tr|}ntd|S)z1convert v1 apt format to v2 (dict in apt_sources)zIDEPRECATION: 'apt_sources' deprecated config key found. Use 'apt' insteadz9apt config: convert V1 to V2 format (source list to dict)rzcloud_config_sources.listzunknown apt_sources format) r:r{r=r|rUr rand_dict_keyr>r-)srclistrsrcentrs r0convert_v1_to_v2_apt_formatrzsGKK '4  7 MNNN " "F''&Az"(2MNNZ(!GCLL " GT " "75666 Nr2cl||d||||<||=dSdS)ziconvert an old key to the new one if the old one exists returns true if a key was found and convertedNTFr<)oldcfgaptcfgoldkeynewkeys r0 convert_keyr"s>zz&$+F++v 6Nt 5r2chgd}d}ddgi}|D]\}}t||||rd}|r|g|d<dSdS)zBconvert old apt_mirror keys into the new more advanced mirror spec)) apt_mirroruri)apt_mirror_searchrf)apt_mirror_search_dns search_dnsFarchesdefaultTrN)r")rrkeymap convertednewmcfgr r!s r0convert_mirrorr.s{F I)%G  vw 7 7 I&$Iy&&r2c ddddddddddd d }g}|D])}||vr#||d vr||=||*|s|Std d ||dd}|wtd|D]X}||}||}||=|||d.|||krt d|d||d|Y|Si}|D]"}||t |||||#t|||D]*}||dt d|z+||d<|S)z:convert old to new keys and adapt restructured mirror specrONproxy http_proxy https_proxy ftp_proxyrNrrR) apt_sourcesr$r&r' apt_proxyapt_http_proxy apt_ftp_proxyapt_https_proxyapt_preserve_sources_listapt_custom_sources_listrR)NrzdDEPRECATION apt: converted deprecated config V2 to V3 format for keys '%s'. Use updated config keys.z, r8zODEPRECATION: apt config: deprecated V1/2 and V3 format specified, preferring V3z3Old and New apt format defined with unequal values z vs z @ z&old apt key '%s' left after conversion)ryr:r{rr<r-r"r.)r mapoldkeys needtoconvertr  newaptcfgr!verifyrs r0convert_v2_to_v3_apt_formatr?s"!!!%&&&%<#12  JM-- V  f~++6NN$$V,,,  KK / -   5$''I     $  F'FF^Fv~vt!>" " " "   %G Nr2ct|||}|dS|dd}|(tj|dd}|&t |dd|||}|S)zpass the three potential stages of mirror specification returns None is neither of them found anything otherwise the first hit is returnedNr%rfr()r\r<r rGrP)r3rIr.r4mcfgrs r0 get_mirrorr__s !j$ 7 7D |tXXeT " "F~'4(@(@AA~& HH\4 ( (*c5   Mr2cT|.tj}td|t |d||}td|t |d||}td|t ||||}|d|d<|S) afind_apt_mirror_info find an apt_mirror given the cfg provided. It can check for separate config of primary and security mirrors If only primary is given security is assumed to be equal to primary If the generic apt_mirror is given that is defining for both Nz!got arch for mirror selection: %srzgot primary mirror: %srzgot security mirror: %srrQ)r r'r:rUr_rX)r3r4r.rTrUrVs r0rWrWys |)++ 5t<<<iu55GII&000j$66GII'111$WgtUCCK( 2K r2cd}fd|D}t|rGtd|tj|d|dznNt j|r/tj |td| ddrEtd|tj| ddSt j|r1tj |td |dSdS) zHapply_apt_config Applies any apt*proxy config from if specified ))r0Acquire::http::Proxy "%s";)r1rb)r3zAcquire::ftp::Proxy "%s";)r2zAcquire::https::Proxy "%s";cpg|]2\}}|||z3SrHr)rrfmtr3s r0rz$apply_apt_config..s;LLL{ccggdmmLsSWWT]]"LLLr2zwrite apt proxy info to %srz#no apt proxy configured, removed %sconfNzwrite apt config info to %sz$no apt config configured, removed %s) rzr:rUr rrrrisfiledel_filer<)r3 proxy_fname config_fnamecfgsproxiess` r0r]r]sH  DMLLLdLLLG 7||F . <<<  TYYw%7%7$%>????  $ $F k""" 7EEE wwvtH />>>  cggfoo66666  % %H l### 8,GGGGGHHr2Tcdfd}fd}|dkr |S|dks|dkr |Std)adapt-key replacement commands implemented: 'add', 'list', 'finger' @param output_file: name of output gpg file (without .gpg or .asc) @param data: key contents @param human_output: list keys formatted for human parsing @param hardened: write keys to to /etc/apt/cloud-init.gpg.d/ (referred to with [signed-by] in sources file) c0tjtrtgng}tjt D]I}|ds|dr|t |zJ|r|ndS)zreturn all apt keys /etc/apt/trusted.gpg (if it exists) and all keyfiles (and symlinks to keyfiles) in /etc/apt/trusted.gpg.d/ are returned based on apt-key implementation z.gpgz.ascr)rrrfAPT_LOCAL_KEYSlistdirAPT_TRUSTED_GPG_DIRrtry) key_filesfiles r0_get_key_fileszapt_key.._get_key_filess)+~(F(FN^$$B J233 = =D}}V$$ = f(=(= =  !4t!;<<<%-yy2-r2cd}s.tjtdn rtnt }t j}d|}tj||nz#tj $r0tjtdYn returns filepath to new keyring, or '/dev/null' when an error occurs z /dev/nullz)Unknown filename, failed to add key: "{}"z{}{}.gpgz Gpg error, failed to add key: {}z#Decode error, failed to add key: {}) r logexcr:r?CLOUD_INIT_GPG_DIRrprdearmorrrrUnicodeDecodeError)rkey_dirstdoutrrrrs r0 apt_key_addzapt_key..apt_key_adds1    K@GGMM     *2K&&8KT**&--g{CC  62222-    ;BB4HH&    >EEdKK sAB.apt_key_lists &(( L LH L M M MNNNN- L L L 98UKKKKKKKK Lyy"""s);A0 A++A0rfingerr|z@apt_key() commands add, list, and finger are currently supported)r-)commandrrrrr}r{rrss ```` @r0rrs . . .8 # # # # # #%{}} H  6 1 1|~~ N   r2z cloud-init)NNr)FN)NFN)NNN)NNFT)N__doc__rrrrdloggingrtextwrapr cloudinitrrrrr cloudinit.cloudr cloudinit.configr cloudinit.config.schemar r cloudinit.settingsr getLogger__name__r:rcrnrprvrrr__annotations__rr_r^rr)r,r(r+r1rr|rDrIrBrvrrArrr\rrrrZr[rMrrr rgrr"r.r?r;rPrXr\r_rWr]rrxrHr2r0rs|100  $$$$$$++++++++++!!!!!!######<<<<<<<<++++++g!!#'/1  X    )6 9$@ ?B B DJ!{^^j^^^@ ,t   ! : : +34 66 6"KKK  C C C C& &&#(&/5&=A& &&&&4%%%- - - `    B 3 3 3 3F$III2---$###L777222<"        &*** FJV V V V r6&&&$HHHV   &&&R%%%0&40HHH:HLN N N N d"r2