
­­­­­­­­­­­­­­­­­­
<!DOCTYPE html>
<html>
3
ie\                @   s  d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlZd dl	Z	d dl
Z
d dlZd dlZd dlZd dlZd dlZd dlZd dlmZmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlmZmZ d d	lmZ d dlZyd dl Z W n e!k
r   Y nX d dl"Z"d dl#Z"d
dl$m%Z%m&Z&m'Z' d
dl(m)Z)m*Z*m+Z+m,Z, d
dl-m.Z.m/Z/m0Z0 d
dl1m2Z2m3Z3 d
dl4T d
dl5m6Z6m7Z7 d
dl8m9Z9m:Z:m;Z;m<Z<m=Z=m>Z> d
dl?m@Z@mAZA d
dlBmCZC d
dl&mDZDmEZE d
dl&mFZFmGZGmHZH d
dl&mIZImJZJmKZK d
dl&mLZL d
dl&mMZM d
dl&mNZN d
dlOmPZPmQZQ d
dlRmSZSmTZTmUZU d
dlVmWZW d
dlXmYZY d
dlZm[Z[ d
dl\m]Z]m^Z^ d
d l_m`Z` d!d"l$maZambZb d!d#l$mcZcmdZdmeZemfZf d!d$l$mgZgmhZhmiZimjZjmkZk d!d%l'mlZl d!d&lmmnZn d!d'l$moZo ejpjqejpjrejpjsetd(Zudd)dd*d)d+d,d-Zvd.d/ Zwyevd0d1d2d3 d4d5gZxW n eyk
rj   d4gZxY nX ejzexd6d7d8 Z{d9d: Z|d;Z}ej~jejpje} d<e} d=d>d? ZG d@dA dAecZG dBdC dCeZejd5exkdDG dEdF dFeZG dGdH dHeZG dIdJ dJeZG dKdL dLeZG dMdN dNeZG dOdP dPeZG dQdR dReZdSdT ZdUdV ZG dWdX dXZG dYdZ dZZd[d\ Zd]d^ Zej~jd_ee j d`da Zej~jdbee.jj dcdd ZdS )e    N)	unhexlify
b2a_base64)ConfigParser)datetime)timezone)	timedelta)sha256)BytesIOStringIO)patch   )xattrhelpersplatform)ArchiveChunkBufferflags_noatimeflags_normal)Archiverparse_storage_quotaPURE_PYTHON_MSGPACK_WARNING)Cache
LocalCache)*)bytes_to_longnum_aes_blocks)KeyfileKeyBaseRepoKey
KeyfileKey
PassphraseTAMRequiredErrorArchiveTAMRequiredError)RepoIdMismatchNotABorgKeyFile)FileIntegrityError)Locationget_security_dir)ManifestMandatoryFeatureUnsupportedArchiveInfo)EXIT_SUCCESSEXIT_WARNING
EXIT_ERROR)
bin_to_hex)MAX_S)msgpack)RstToTextLazyrst_to_terminal)	IECommandPatternMatcherparse_pattern)Item)
LockFailed)setup_logging)RemoteRepositoryPathNotAllowed)
Repository   )has_lchflags
has_llfuse)BaseTestCase	changedirenvironment_variable
no_selinux)are_symlinks_supportedare_hardlinks_supportedare_fifos_supportedis_utime_fully_supportedis_birthtime_fully_supported)fakeroot_detected)make_attic_repo)keyz..F    )archiverforkexeinputbinary_outputc             O   s  |ryX|d krt jddf}n$t|tr.|f}nt|ts@tdtj|| tj|d}d}	W nZ tj	k
r }
 z|
j
}|
j}	W Y d d }
~
X n, tk
r }
 zd}|
j}	W Y d d }
~
X nX |r|	|fS |	tj|fS nt jt jt j  }}}zt|j t _t|t j_t }tj|dd } t _t _| d kr8t } d	d
 | _t| _tt_y| jt|}W nF tk
r }
 z(|j   |
j|r|j! n
|j! j fS d }
~
X nX | j"|}	|j   |	|r|j! n
|j! j fS |||  t _t _t _X d S )Nz-mzborg.archiverz"exe must be None, a tuple or a str)stderrrN   r    zutf-8)encodingc              W   s   d S )N )argsrS   rS    /usr/lib64/python3.6/archiver.py<lambda>d   s    zexec_cmd.<locals>.<lambda>)#sys
executable
isinstancestrtuple
ValueError
subprocessZcheck_outputZSTDOUTZCalledProcessErroroutput
returncode
SystemExitcodeosfsdecodestdinstdoutrP   r
   decoder	   bufferioTextIOWrapperr   Zprerun_checksr*   	exit_coder   
parse_argslistflushgetvalueZrun)rK   rL   rM   rN   rO   rT   kwborgr^   reterd   re   rP   Zoutput_textrS   rS   rU   exec_cmdE   sN    



,
rs   c              C   s6   t jdsdS tjddgtjd} | j \}}d|kS )NtarFz	--version)re   s   GNU tar)shutilwhichr]   PopenPIPEZcommunicate)popenre   rP   rS   rS   rU   have_gnutarv   s
    
rz   helpzborg.exeT)rM   rL   pythonbinary)Zparamsc                s8   | j dkrd  n| j dkr d ntd fdd}|S )Nr|   r}   zborg.exez"param must be 'python' or 'binary'c                 s   t |  dd|S )NT)rM   rL   )rs   )rT   ro   )rM   rS   rU   exec_fn   s    zcmd.<locals>.exec_fn)Zparamr\   )Zrequestr~   rS   )rM   rU   cmd   s    

r   c             C   s   |j d}|j d}|j d}|jdjd | dddt| \}}| d	d
| t|\}}tt| | dd
| \}}W d Q R X | dd
| d\}}| d	d
| t|\}}d S )NreporN   r^   Z	test_filecontentinitz--encryption=nonez%screatez%s::archiveextractzdoes/not/match)mkdirjoinwriterZ   r?   )r   tmpdirr   rN   r^   rcoutrS   rS   rU   test_return_codes   s    



r   z/tmp/borg-mountzneeds a 16MB fs mounted on %s)reasonc          )   C   s  ddd}t dd t}tjj|d}tjj|d}tjj|d}xtd	D ]x}tj|dd
 tj|dd
 ||dddd | d|\}}|tkrt	d|| zd\}	}
x|	rn|
d7 }
y||dd W n4 t
k
r } z|jtjkrP  W Y d d }~X nX z6| dd||
f |\}}|tk}	|	s:t	d|| W d tjtjj|ddd
 tjtjj|d X qW W d tj|dd
 X | d|\}}|tkrt	d|| | dd|\}}|tkrRt	d|| qRW W d Q R X d S )NTc          
   S   s   t j| dd tj|  |r<tjd|}|dkr<tjd|}xLt|D ]@}tjj| d| }t	|d}tj
|}|j| W d Q R X qFW d S )NT)ignore_errorsr;   zfile%03dwb)ru   rmtreerb   r   randomZrandintrangepathr   openurandomr   )dircountsizerndifnfdatarS   rS   rU   
make_files   s    

z"test_disk_full.<locals>.make_filesYES)!BORG_CHECK_I_KNOW_WHAT_I_AM_DOINGr   rN   reserved   )r   P   i F)r   r   r   r;      i@ r   z%s::test%03dzlock.exclusivezlock.rosterrl   checkz--repair)T)Tr   )r@   DF_MOUNTrb   r   r   r   ru   r   r*   printOSErrorerrnoZENOSPCremove)r   r   mountr   rN   r   jr   r   successr   errrS   rS   rU   test_disk_full   sL    

r   c               @   sZ   e Zd ZdZdZdZdd Zdd Zdd	 Zd
d Z	dd Z
dd ZdddZdd ZdS )ArchiverTestCaseBaseNFrQ   c             C   s  dt jd< dt jd< dt jd< dt jd< | j r6t p8d | _tj | _t jj	| jd| _
| j| j
 | _t jj	| jd	| _t jj	| jd
| _t jj	| jd| _t jj	| jd| _t jj	| jd| _t jj	| jd| _| jt jd< | jt jd< t j| j t j| jd t j| j t j| j t j| j t| jd}|jd W d Q R X t| jd}|jd W d Q R X t j | _t j| j d S )Nr   r   "BORG_DELETE_I_KNOW_WHAT_I_AM_DOINGZwaytooeasyonlyfortestsBORG_PASSPHRASEZdisabledZBORG_SELFTEST
repositoryrN   r^   keyscacheZexcludespatternsZBORG_KEYS_DIRZBORG_CACHE_DIRi  r   s1   input/file2
# A comment line, then a blank line

sI   +input/file_important
- input/file*
# A comment line, then a blank line

)rb   environFORK_DEFAULTr   rK   tempfileZmkdtempr   r   r   repository_pathprefixrepository_location
input_pathoutput_path	keys_path
cache_pathexclude_file_pathpatterns_file_pathr   chmodr   r   getcwd_old_wdchdir)selffdrS   rS   rU   setUp   s6    





zArchiverTestCaseBase.setUpc             C   s&   t j| j tj| jdd t  d S )NT)r   )rb   r   r   ru   r   r   r7   )r   rS   rS   rU   tearDown  s    zArchiverTestCaseBase.tearDownc       	         s   |j dd}|j dd }|jdd}|d kr2| j}t||| j| jd|\}}||kr`t| | j|| |rxtj	 nt |rdnd}|j
 fd	d
|jddD }|S )Nrj   r   rL   rO   F)rL   rM   rK   rJ   rQ   c             3   s   | ]} |kr|V  qd S )NrS   ).0line)pp_msgrS   rU   	<genexpr>,  s    z+ArchiverTestCaseBase.cmd.<locals>.<genexpr>T)keepends)popgetr   rs   EXErK   r   assert_equalr   encoder   
splitlines)	r   rT   ro   rj   rL   rO   rq   r^   emptyrS   )r   rU   r     s     zArchiverTestCaseBase.cmdc             C   s   | j dd| jd | t d S )Nr   z--compression=lz4z::)r   r   src_dir)r   namerS   rS   rU   create_src_archive0  s    z'ArchiverTestCaseBase.create_src_archivec             C   sF   t | jdd}|& tj|tj\}}t||||}W d Q R X ||fS )NT)	exclusive)r:   r   r'   loadNO_OPERATION_CHECKr   )r   r   r   manifestrI   archiverS   rS   rU   open_archive3  s
    z!ArchiverTestCaseBase.open_archivec             C   s   t | jddS )NT)r   )r:   r   )r   rS   rS   rU   open_repository:  s    z$ArchiverTestCaseBase.open_repositoryr   c          	   C   sj   t jj| j|}t jjt jj|s6t jt jj| t|d }|d krRd| }|j| W d Q R X d S )Nr      X)	rb   r   r   r   existsdirnamemakedirsr   r   )r   r   r   contentsfilenamer   rS   rS   rU   create_regular_file=  s    z(ArchiverTestCaseBase.create_regular_filec             C   s  | j dd#d | j ddd | j dd$d tjdd t r`tjtjj| jdtjj| jd	 t r~tj	d
tjj| jd | j ddd t
j rt
j| jrt
jtjj| jddd t
jtjj| jddd t rtjtjj| jd trtjtjj| jdtj yZtjddtjB tjdd tjddtjB tjdd tjdd tjddd d}W nV tk
r   d}Y n> tk
r } z |jtjtjfkr d}W Y d d }~X nX tj d | j d!d"d |S )%zFCreate a minimal test case including all supported file types
        file1i   r   )r   flagfilez
dir2/file2zinput/file1i	  hardlinkZ	somewherelink1	fusexattrr;   zuser.foos   barz
user.emptyrJ   fifo1z
input/bdevi  
   r   z
input/cdev   (   z
input/dir2im  r      TFNr   r   i @ i @ )!r   rb   r   rC   linkr   r   r   rB   symlinkr   XATTR_FAKEROOT
is_enabledsetxattrrD   mkfifor<   r   	set_flagsstat	UF_NODUMPmknodS_IFBLKmakedevS_IFCHRchownPermissionErrorr   r   ZEINVALZENOSYStimesleep)r   	have_rootrr   rS   rS   rU   create_test_filesG  s@    
z&ArchiverTestCaseBase.create_test_files)r   N)__name__
__module____qualname__r   r   r   r   r   r   r   r   r   r   r  rS   rS   rS   rU   r      s   

r   c               @   s  e Zd Zejje  ddZdd Zdd Z	edd Z
d	d
 Zdd Zejje  dddd Zejje  dddd Zejje  ddejje  dddd Zejje  ddejje  dddd Zdd Zdd Zdd Zdd Zd d! Zd"d# Zd$d% Zd&d' Zd(d) Zd*d+ Zd,d- Zd.d/ Zd0d1 Z d2d3 Z!ee"j#e$d4d5d6 Z%ed7d8 Z&ed9d: Z'ed;d< Z(d=d> Z)d?d@ Z*dAdB Z+dCdD Z,dEdF Z-dGdH Z.dIdJ Z/dKdL Z0dMdN Z1dOdP Z2dQdR Z3dSdT Z4dUdV Z5dWdX Z6dYdZ Z7d[d\ Z8d]d^ Z9d_d` Z:dadb Z;dcdd Z<dedf Z=dgdh Z>didj Z?dkdl Z@dmdn ZAdodp ZBdqdr ZCdsdt ZDdudv ZEejje  dddwdx ZFejjeGjH dyddzd{ ZIejjeGjH d|dd}d~ ZJdd ZKdd ZLdd ZMdd ZNdd ZOdd ZPdd ZQdd ZRdd ZSdd ZTdd ZUdd ZVdd ZWdd ZXdd ZYdd ZZdd Z[dd Z\dd Z]dd Z^dd Z_e"j#e$d4dd Z`ejjdeajbkdddd Zcdd Zddd Zedd Zfdd Zgdd Zhdd Zidd Zjdd Zkdd Zle"j#e$d4dd ZmejjnddĄ ZoddƄ ZpddȄ Zqddʄ Zrdd̄ Zsdd΄ ZtddЄ Zudd҄ ZvddԄ Zwddք Zxdd؄ Zyddڄ Zzdd܄ Z{ddބ Z|dd Z}dd Z~dd Zdd Zdd Zdd Zdd Zdd Zdd Zdd Zdd Zdd ZdddZdd Zdd Zdd Zd d Zdd Zdd Zdd Zdd	 Zd
d Zdd Zdd Zdd Zdd Zdd Ze"j#e$d4dd Ze"j#e$d4dd Ze"j#e$d4dd Ze"j#e$d4dd Zdd Zd d! Zd"d# Zd$d% Zd&d' Zd(d) Zd*d+ Zd,d- Zd.d/ Zd0d1 Zd2d3 Zd4d5 Zd6d7 Zd8d9 Zd:d; Zejje  ddd<d= Zd>d? Zd@dA ZdBdC ZdDdE ZdFdG ZdHdI ZdJdK ZdLdM ZdNdO ZdPdQ ZdRdS ZdTdU ZdVdW ZdXdY ZdZd[ Zd\d] Zd^d_ Zd`da Zdbdc Zddde Zdfdg Zejje  dhdZejjejĐdi djdZedkdl ZeeŐdmdn Zedodp Zeedqdr Zeedsdt Zʐdudv ZejjeGjH d|ddwdx ZejjeGjH d|ddydz Z͐d{d| Zΐd}d~ ZϐdS (  ArchiverTestCasezhardlinks not supported)r   c             C   s   t | j| j}t|S )N)r-   _extract_repository_idr   r&   )r   repository_idrS   rS   rU   r&     s    z!ArchiverTestCase.get_security_dirc       
      C   s  | j  }| jdddd| jdd}| jd| | jd| | jd	d
| jd d | jd	d
d| jd d}| jd| | jd| td | jd| jd  W d Q R X | jdd| j}| jd| | jd| ddddddddg}t r|jd t r|jd t r|jd  |s.|j	d |j	d t
rR|j	d tj	tjjdd! | jdd| jd }x|D ]}| j|| qlW | jdd" | jd#| jd }t
rd$nd%}| jd&| | tj| j | jd#| jd }d'd( }	| j|	||	| d S ))Nr   z--encryption=repokeyz--show-versionz	--show-rcT)rL   zborgbackup versionz%terminating with success status, rc 0r   z--exclude-nodumpz::testrN   z--statsz::test.2zArchive name: test.2zThis archive: r^   r   rl   z--shorttestztest.2z
input/bdevz
input/cdevz
input/dir2zinput/dir2/file2zinput/emptyzinput/file1zinput/flagfilezinput/fifo1zinput/link1zinput/hardlinkr   zoutput/inputinfo      zNumber of files: %dc             S   sR   ddddddg}g }x2| j  D ]&}x |D ]}|j|r(|j| q(W qW dj|S )NzName:zFingerprint:zNumber of files:zThis archive:zAll archives:zChunk index:
)r   
startswithappendr   )r^   prefixesresultr   r   rS   rS   rU   filter  s    

z9ArchiverTestCase.test_basic_functionality.<locals>.filter)r  r   r   	assert_inr?   rD   r  rB   rC   r   r<   rb   r   r   assert_dirs_equalru   r   r   r   )
r   r  r^   Zlist_outputexpectedr   Zinfo_outputZ
item_countZinfo_output2r  rS   rS   rU   test_basic_functionality  sX    







z)ArchiverTestCase.test_basic_functionalityc             C   s   t jj| jd}t jj| jd}t j| t j| t jj|d}t jj|d}| j|dd t j|| | jdd| j | jd| jd	 d
d
 | jdd| jd	 }dd |j	dD }d S )Nabr   s   123456)r   r   z--encryption=noner   z::testrN   rl   z--json-linesc             S   s   g | ]}|rt j|d  qS )r   )jsonloads)r   r   rS   rS   rU   
<listcomp>  s    z?ArchiverTestCase.test_create_duplicate_root.<locals>.<listcomp>r  )
rb   r   r   r   r   r   r   r   r   split)r   path_apath_bhl_ahl_barchive_listpathsrS   rS   rU   test_create_duplicate_root  s    

z+ArchiverTestCase.test_create_duplicate_rootc             C   sd   t jj| jdd}t jj|d}| j| }tjtj | j	dd| W d Q R X | j	ddd| d S )NZparent1Zparent2r   r   z--encryption=nonez--make-parent-dirs)
rb   r   r   r   r   pytestraisesr:   ZParentPathDoesNotExistr   )r   parent_pathr   r   rS   rS   rU   test_init_parent_dirs  s    
z&ArchiverTestCase.test_init_parent_dirsc             C   s   | j dd| j y*tjtjtj}|jtjj| j	d W nP t
k
r } z4|jtjkrdtjd n|jtjkrztjd W Y d d }~X nX | j d| jd d |j  td	 | j d
| jd  W d Q R X d S )Nr   z--encryption=repokeyzunix-socketz&unix sockets disabled or not supportedz(permission denied to create unix socketsr   z::testrN   r^   r   )r   r   socketZAF_UNIXZSOCK_STREAMZbindrb   r   r   r   r   r   ZEPERMr%  skipEACCEScloser?   )r   Zsockr   rS   rS   rU   test_unix_socket  s    
z!ArchiverTestCase.test_unix_socketzsymlinks not supportedc             C   sV   | j   | jdd| j | jd| jd d td | jd| jd  W d Q R X d S )Nr   z--encryption=repokeyr   z::testrN   r^   r   )r  r   r   r?   )r   rS   rS   rU   test_symlink_extract  s    
z%ArchiverTestCase.test_symlink_extractz4cannot properly setup and execute test without utimec             C   s   dd }| j   d\}}|d}tjd||f | jdd| j | jd| jd	 d
 td | jd| jd	  W d Q R X tjd}tjd}|rn d S )Nc             S   sn   t j| j}y(tt j| t}|j  W d Q R X W n tk
rH   dS X t j| j}ttk}|oh||kS d S )NF)rb   r   st_atime_nsr   r   readr   r   )	some_fileatime_beforefileatime_afternoatime_usedrS   rS   rU   has_noatime  s    z0ArchiverTestCase.test_atime.<locals>.has_noatime[8zinput/file1r   z--encryption=repokeyr   z::testrN   r^   r   zoutput/input/file1)r7  r8  )r  rb   utimer   r   r?   r   )r   r6  atimemtimehave_noatimestistorS   rS   rU   
test_atime  s    


zArchiverTestCase.test_atimez8cannot properly setup and execute test without birthtimec             C   s   | j   d\}}}tjd||f tjd||f | jdd| j | jd| jd d	 td
 | jd| jd  W d Q R X tjd}tjd}d S )N k8Cm8 n8zinput/file1r   z--encryption=repokeyr   z::testrN   r^   r   zoutput/input/file1)r@  rA  rB  )r  rb   r9  r   r   r?   r   )r   	birthtimer;  r:  r=  r>  rS   rS   rU   test_birthtime  s    



zArchiverTestCase.test_birthtimec             C   s   | j   d\}}}tjd||f tjd||f | jdd| j | jdd| jd	 d
 td | jd| jd	  W d Q R X tjd}tjd}d S )N k8Cm8 n8zinput/file1r   z--encryption=repokeyr   z--nobirthtimez::testrN   r^   r   zoutput/input/file1)rE  rF  rG  )r  rb   r9  r   r   r?   r   )r   rC  r;  r:  r=  r>  rS   rS   rU   test_nobirthtime.  s    



z!ArchiverTestCase.test_nobirthtimec          	   C   s   t | j}|jS Q R X d S )N)r:   r   id)r   r   r   rS   rS   rU   r  ?  s    z'ArchiverTestCase._extract_repository_idc             C   sv   t d d}|jtjj|d |jddt| ttjj|dd}|j| W d Q R X t	| j
}|jS Q R X d S )N)interpolationconfigr   rI  w)r   r0  rb   r   r   setr-   r   r   r:   r   rI  )r   r   rI  rK  r   r   rS   rS   rU   _set_repository_idC  s    
z#ArchiverTestCase._set_repository_idc       	      C   s\  dd }t jj| jd}d}ddt>  }|t| | }t|d:}|j|d |j| |j|d |j	 }|j
| W d Q R X ||||}|rX| jdd	| j | jd
| jd d t| j | jdd| jd  W d Q R X | jdd t jj| jdd}t|dH}| j|j|d|  | j|jt|| | j|j|d|  W d Q R X d S )Nc             S   s   t j| }d}|r0t|dr0|jd |jkr0d}|rtt drtt drt| dP}y0|jdt jdkrld}|jdt j|krd}W n t	k
r   d}Y nX W d Q R X |S )	NT	st_blocksi   F	SEEK_HOLE	SEEK_DATArbr   )
rb   r   hasattrrO  st_sizer   seekrP  rQ  r   )r   
total_size	hole_sizestsparser   rS   rS   rU   	is_sparseM  s    
z4ArchiverTestCase.test_sparse_file.<locals>.is_sparserY  s   foobarr  r;   r   r   z--encryption=repokeyr   z::testrN   r   z--sparsezoutput/inputrR      )rb   r   r   r   ZCHUNK_MAX_EXPlenr   rU  r   telltruncater   r   r?   r   r  r   r0  )	r   rZ  r   r   rW  rV  r   posZsparse_supportrS   rS   rU   test_sparse_fileL  s0    
 z!ArchiverTestCase.test_sparse_filec             C   s   dddg}x2|D ]*}t jj| j|}t|d W d Q R X qW | jdd| j | jd| jd d	 x>|D ]6}td
$ | jd| jd t jjd	| W d Q R X qhW d S )NZnormalzwith some blanksz(with_parens)r   r   z--encryption=repokeyr   z::testrN   r^   r   )rb   r   r   r   r   r   r   r?   )r   	filenamesr   rS   rS   rU   test_unusual_filenames  s    



(z'ArchiverTestCase.test_unusual_filenamesc             C   s   | j   dtjd< | jdd| j | j| j}| jd| jd d tj| j | jdd| j | j	| j| | j
|| j| j | jr| jd| jd	 dtd
 n,tjtj | jd| jd	 d W d Q R X d S )N
passphraser   r   z--encryption=repokeyr   z::testrN   z--encryption=nonez::test.2)rj   )r  rb   r   r   r   r  r   ru   r   rN  r   r   r,   r%  r&  r   EncryptionMethodMismatch)r   r	  rS   rS   rU   test_repository_swap_detection  s    
z/ArchiverTestCase.test_repository_swap_detectionc             C   s   | j   | jdd| jd  dtjd< | jdd| jd  | jd| jd	 d
 tj| jd  tj| jd | jd  | j	r| jd| jd d
t
d n,tjtj | jd| jd d
 W d Q R X d S )Nr   z--encryption=none_unencryptedrc  r   z--encryption=repokey
_encryptedr   z_encrypted::testrN   z_encrypted::test.2)rj   )r  r   r   rb   r   ru   r   r   renamer   r,   r%  r&  r   RepositoryAccessAborted)r   rS   rS   rU   test_repository_swap_detection2  s    
z0ArchiverTestCase.test_repository_swap_detection2c             C   s   | j   dtjd< | jdd| j | j| j}| jd| jd d tj| j | jdd| j | j	| j| | j
|| j| j | jd	d
| j | jr| jd| jd dtd n,tjtj | jd| jd d W d Q R X d S )Nrc  r   r   z--encryption=repokeyr   z::testrN   z--encryption=nonedeletez--cache-onlyz::test.2)rj   )r  rb   r   r   r   r  r   ru   r   rN  r   r   r,   r%  r&  r   rd  )r   r	  rS   rS   rU   'test_repository_swap_detection_no_cache  s    
z8ArchiverTestCase.test_repository_swap_detection_no_cachec             C   s   | j   | jdd| jd  dtjd< | jdd| jd  | jd| jd	 d
 | jdd| jd  | jdd| jd  tj| jd  tj| jd | jd  | j	r| jd| jd d
t
d n,tjtj | jd| jd d
 W d Q R X d S )Nr   z--encryption=nonerf  rc  r   z--encryption=repokeyrg  r   z_encrypted::testrN   rk  z--cache-onlyz_encrypted::test.2)rj   )r  r   r   rb   r   ru   r   r   rh  r   r,   r%  r&  r   ri  )r   rS   rS   rU   (test_repository_swap_detection2_no_cache  s    
z9ArchiverTestCase.test_repository_swap_detection2_no_cachec             C   s   | j   | jdd| j | jd| jd d tj| j tdd4 | jdd| j | jdd	| j tj| j  W d Q R X td dR | jr| jd| jd
 dt	d n,t
jtj | jd| jd
 d W d Q R X W d Q R X d S )Nr   z--encryption=repokeyr   z::testrN   rQ   )r   rk  z--cache-onlyz::test.2)rj   )r  r   r   ru   r   r   r@   r&   r   r,   r%  r&  r   CacheInitAbortedError)r   rS   rS   rU   7test_repository_swap_detection_repokey_blank_passphrase  s    zHArchiverTestCase.test_repository_swap_detection_repokey_blank_passphrasec             C   s   | j dd| j | j }tj| j| jd  tdd | j d| jd  W d Q R X ttjj	|d}|j
 }W d Q R X | j d| jd  tj| j | j d| jd  tj| | j d| jd  xd
D ]}qW d S )Nr   z--encryption=repokeyZ_newyes)Z BORG_RELOCATED_REPO_ACCESS_IS_OKr  locationkey-typemanifest-timestamp)rq  rr  rs  )r   r   r&   rb   rh  r   r@   r   r   r   r0  ru   r   r   )r   Zsecurity_dirr   rq  r3  rS   rS   rU   test_repository_move  s    


z%ArchiverTestCase.test_repository_movec          
   C   sP   | j dd| j ttjj| j dd}|jd W d Q R X | j d| j d S )Nr   z--encryption=repokeyrq  rL  zsomething outdatedr  )r   r   r   rb   r   r   r&   r   )r   r   rS   rS   rU   test_security_dir_compat  s    z)ArchiverTestCase.test_security_dir_compatc             C   s   | j dd| j | j d| j tj| j | j d| j tj| j tj| j  | jrl| j d| jtd n&tj	t
j | j d| j W d Q R X tdd | j d| j W d Q R X d S )Nr   z--encryption=noner  )rj   rp  )Z*BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK)r   r   ru   r   r   r&   r   r,   r%  r&  r   rn  r@   )r   rS   rS   rU   test_unknown_unencrypted  s    z)ArchiverTestCase.test_unknown_unencryptedc          !   C   s   | j dd| j | jd | j d| jd d td | j d| jd d	d
 | jd | j d| jd d	d W d Q R X | jd | j d| jd d	d W d Q R X | jd | j d| jd d	d W d Q R X W d Q R X d S )Nr   z--encryption=repokeyzdir/filer   z::testrN   r^   r   z--strip-components3r3  21zinput/dir/file0)r   r   r   r?   assert_creates_file)r   rS   rS   rU   test_strip_components  s    

  z&ArchiverTestCase.test_strip_componentsc             C   s   t jt jj| jd t jt jj| jd | jddd t jt jj| jdt jj| jd t jt jj| jdt jj| jd t jt jj| jdt jj| jd | jd	 t jt jj| jd	t jj| jd
 | jdd| j | jd| jd d d S )NZdir1zdir1/subdirsources   123456)r   Zabbazdir1/hardlinkzdir1/subdir/hardlinkzdir1/source2z	dir1/aaaar   z--encryption=repokeyr   z::testrN   )	rb   r   r   r   r   r   r   r   r   )r   rS   rS   rU   _extract_hardlinks_setup  s    
z)ArchiverTestCase._extract_hardlinks_setupzllfuse not installedc          ,   C   s   | j   tjj| jd}tjdkr,ddg}nddg}| j| jd |df|  t	| W d Q R X W d Q R X | j| jd |df|  t	| W d Q R X W d Q R X | j| jd |f|  t	| W d Q R X W d Q R X d S )	N
mountpointdarwinz-oz$ignore_permissions,defer_permissionsZignore_permissionsz::testz--strip-components=2z
input/dir1)
r~  rb   r   r   r   rW   r   
fuse_mountr   r?   )r   r  Zignore_permsrS   rS   rU   test_fuse_mount_hardlinks1  s    




z*ArchiverTestCase.test_fuse_mount_hardlinksc             C   s2   | j   td | jd| jd  W d Q R X d S )Nr^   r   z::test)r~  r?   r   r   )r   rS   rS   rU   test_extract_hardlinks1S  s    
z(ArchiverTestCase.test_extract_hardlinks1c             C   s^   | j   td | jd| jd dd W d Q R X td | jd| jd d W d Q R X d S )Nr^   r   z::testz--strip-componentsrx  z
input/dir1)r~  r?   r   r   )r   rS   rS   rU   test_extract_hardlinks2^  s    


z(ArchiverTestCase.test_extract_hardlinks2c             C   s   t jj| jd}t jj| jd}t j| t j| t jj|d}t jj|d}| j|dd t j|| | jdd| j | jd| jd	 d
d
 t	d | jd| jd	  W d Q R X d S )Nr  r  r   s   123456)r   r   z--encryption=noner   z::testrN   r^   r   )
rb   r   r   r   r   r   r   r   r   r?   )r   r  r  r   r!  rS   rS   rU   test_extract_hardlinks_twicep  s    


z-ArchiverTestCase.test_extract_hardlinks_twicec             C   s,  | j dd| j | jddd | jddd | jddd | jd	dd | j d
d| jd d td | j d| jd d W d Q R X | jttjddg td | j dd| jd  W d Q R X | jttjdddg td  | j dd| j | jd  W d Q R X | jttjdddg d S )Nr   z--encryption=repokeyr   i   r   )r   file2file3file4r   z--exclude=input/file4z::testrN   r^   r   zinput/file1zoutput/inputz--exclude=input/file2z--exclude-from=i @ i @ i @ i @ )	r   r   r   r?   r   sortedrb   listdirr   )r   rS   rS   rU   test_extract_include_exclude  s    


$z-ArchiverTestCase.test_extract_include_excludec          #   C   s  | j dd| j | jddd | jddd | jddd | jd	dd | jd
dd | j dd| jd d td | j d| jd  W d Q R X | jttjddddd
g tj	d td | j dd| jd  W d Q R X | jttjdddg tj	d td | j ddd| jd  W d Q R X | jttjddd
g tj	d td$ | j dd| j
 dd| jd  W d Q R X | jttjddg d S )Nr   z--encryption=repokeyr   i   r   )r   r  r  r  file333r   z--exclude=re:input/file4$z::testrN   r^   r   zoutput/inputz--exclude=re:file3+z--exclude=input/file2z--exclude=re:file[01]z--exclude-from=z--exclude=re:file1z--exclude=re:file(\d)\1\1$i @ i @ i @ i @ i @ )r   r   r   r?   r   r  rb   r  ru   r   r   )r   rS   rS   rU   "test_extract_include_exclude_regex  s.    




 

z3ArchiverTestCase.test_extract_include_exclude_regexc          /   C   s  | j dd| j | jddd | jddd | jddd | jd	dd | jd
dd | jdd d t| jd}|jd |jd W d Q R X | j dd| j | jd d td | j d| jd  W d Q R X | jtt	j
ddddd
g tjd t| jd}|jd W d Q R X td  | j dd| j | jd  W d Q R X | jtt	j
dddg tjd t| jd.}|jd |jd |jd |jd W d Q R X td  | j dd| j | jd  W d Q R X | jtt	j
ddg d S )!Nr   z--encryption=repokeyr   i   r   )r   r  r  r  r  zaa:somethingr   s   re:input/file4$
s   fm:*aa:*thing
r   z--exclude-from=z::testrN   r^   r   zoutput/inputs
   re:file3+
s   re:file(\d)\1\1$
s   fm:nothingwillmatchthis
s   */file1
s
   re:file2$
i @ i @ i @ i @ i @ i @ )r   r   r   r   r   r   r?   r   r  rb   r  ru   r   )r   r   rS   rS   rU   ,test_extract_include_exclude_regex_from_file  s:    



$




$z=ArchiverTestCase.test_extract_include_exclude_regex_from_filec          "   C   s  | j dd| j | jddd | jddd | jddd | jd	dd | jd
dd | j d| jd d td | j d| jd d W d Q R X | jttjddddd
d	g tj	d td | j dd| jd d W d Q R X | jttjdddg tj	d td | j dd| jd d W d Q R X | jttjddg tj	d td | j d| jd ddd W d Q R X | jttjdddd
g d S )Nr   z--encryption=repokeyr   i   r   )r   r  r  r  r  r   z::testrN   r^   r   zre:.*zoutput/inputz--exclude=re:file[34]$z
re:file\d$z--exclude=input/file1zre:file[12]$zfm:input/file1zfm:*file33*zinput/file2i @ i @ i @ i @ i @ )
r   r   r   r?   r   r  rb   r  ru   r   )r   rS   rS   rU   test_extract_with_pattern  s,    
 

 

 

"z*ArchiverTestCase.test_extract_with_patternc          "   C   s$  | j dd| j | jddd | j d| jd d	 td
 | j d| jd }W d Q R X | jd| tjd td
 | j dd| jd }W d Q R X | jd| tjd td
 | j dd| jd }W d Q R X | jd| tjd td
 | j ddd| jd }W d Q R X | jd| d S )Nr   z--encryption=repokeyr3  i   r   )r   r   z::testrN   r^   r   z
input/filezoutput/inputz--infoz--listi @ )r   r   r   r?   assert_not_inru   r   r  )r   r^   rS   rS   rU   test_extract_list_output  s$    






 z)ArchiverTestCase.test_extract_list_outputc             C   s^   | j dd| j | jddd | j d| jd d	 td
 | j d| jd d}W d Q R X d S )Nr   z--encryption=repokeyr3  i   r   )r   r   z::testrN   r^   r   z
--progressi @ )r   r   r   r?   )r   r^   rS   rS   rU   test_extract_progress  s    
z&ArchiverTestCase.test_extract_progressc             C   s   | j dd| j | jddd | jdt td d	 | jd
t dd	 tjd t rltjdt dt  n| jdt td d	 d S )Nr   z--encryption=repokeyr   i   r   )r   z	cache1/%ss    extra stuff)r   z	cache2/%ss   invalid signaturezinput/cache3zinput/cache1/%szinput/cache3/%sz	cache3/%si @ )	r   r   r   CACHE_TAG_NAMECACHE_TAG_CONTENTSrb   r   rC   r   )r   rS   rS   rU   _create_test_caches$  s    



z$ArchiverTestCase._create_test_cachesc             C   sb   | j dd| j d}| j d| jd d|d tj| j dd	| jd }| j d
d| jd dd}d S )Nr   z--encryption=repokeys    foo

bar
   
r   z::test-)rN   rl   z--json-linesr   z--stdoutT)rO   )r   r   r  r  )r   Z
input_dataitemZextracted_datarS   rS   rU   test_create_stdin2  s    z"ArchiverTestCase.test_create_stdinc             C   s*   | j dd| j | j d| jd dd dS )ztest create without a rootr   z--encryption=repokeyr   z::testr   )rj   N)r   r   )r   rS   rS   rU   test_create_without_root>  s    z)ArchiverTestCase.test_create_without_rootc             C   s`   | j dd| j | jddd | jddd | j dd	d
d| jd }| jd| | jd| dS )z$test create with only a root patternr   z--encryption=repokeyr   i   r   )r   r  r   z-vz--listz--pattern=R inputz::testzA input/file1zA input/file2Ni @ i @ )r   r   r   r  )r   r^   rS   rS   rU   test_create_pattern_rootC  s    z)ArchiverTestCase.test_create_pattern_rootc             C   s~   | j dd| j | jddd | jddd | jddd | j d	d
ddd| jd d}| jd| | jd| | jd| dS )z test file patterns during creater   z--encryption=repokeyr   i   r   )r   r  file_importantr   z-vz--listz--pattern=+input/file_importantz--pattern=-input/file*z::testrN   zA input/file_importantzx input/file1zx input/file2Ni @ i @ i @ )r   r   r   r  )r   r^   rS   rS   rU   test_create_patternL  s    
z$ArchiverTestCase.test_create_patternc             C   s   | j dd| j | jddd | jddd | jddd | jd	dd | j d
dddd| j | jd d}| jd| | jd| | jd| | jd| dS )z test file patterns during creater   z--encryption=repokeyr   i   r   )r   r  Z	otherfiler  r   z-vz--listz--pattern=-input/otherfilez--patterns-from=z::testrN   zA input/file_importantzx input/file1zx input/file2zx input/otherfileNi @ i @ i @ i @ )r   r   r   r   r  )r   r^   rS   rS   rU   test_create_pattern_fileY  s    

z)ArchiverTestCase.test_create_pattern_filec             C   s   t jj| jd| _t| jd}|jd W dQ R X | jdd| j | j	ddd
 | j	ddd
 | j	ddd
 | jdddd| j | jd d}| j
d| | j
d| | j
d| dS )z?test when patterns exclude a parent folder, but include a child	patterns2r   s   + input/x/b
- input/x*
Nr   z--encryption=repokeyz	x/a/foo_ai   r   )r   z	x/b/foo_bzy/foo_yr   z-vz--listz--patterns-from=z::testrN   zx input/x/a/foo_azA input/x/b/foo_bzA input/y/foo_yi @ i @ i @ )rb   r   r   r   patterns_file_path2r   r   r   r   r   r  )r   r   r^   rS   rS   rU   .test_create_pattern_exclude_folder_but_recurseh  s    
z?ArchiverTestCase.test_create_pattern_exclude_folder_but_recursec             C   s   t jj| jd| _t| jd}|jd W dQ R X | jdd| j | j	ddd
 | j	ddd
 | j	ddd
 | jdddd| j | jd d}| j
d| | j
d| | jd| dS )zCtest when patterns exclude a parent folder and, but include a childr  r   s   + input/x/b
! input/x*
Nr   z--encryption=repokeyz	x/a/foo_ai   r   )r   z	x/b/foo_bzy/foo_yr   z-vz--listz--patterns-from=z::testrN   zinput/x/a/foo_az	input/x/azA input/y/foo_yi @ i @ i @ )rb   r   r   r   r  r   r   r   r   r   r  r  )r   r   r^   rS   rS   rU   -test_create_pattern_exclude_folder_no_recursey  s    
z>ArchiverTestCase.test_create_pattern_exclude_folder_no_recursec             C   s   t jj| jd| _t| jd}|jd W dQ R X | jdd| j | j	ddd
 | j	ddd
 t
d" | jdd| j | jd d W dQ R X | jddd| jd }|j }| jd| | jd| dS )zetest that intermediate folders appear first when patterns exclude a parent folder but include a childr  r   s#   + input/x/a
+ input/x/b
- input/x*
Nr   z--encryption=repokeyz	x/a/foo_ai   r   )r   z	x/b/foo_brN   r   z--patterns-from=z::test.rl   z--formatz{type} {path}{NL}zd x/azd x/bi @ i @ )rb   r   r   r   r  r   r   r   r   r   r?   r   r  )r   r   r   Zout_listrS   rS   rU   .test_create_pattern_intermediate_folders_first  s    
z?ArchiverTestCase.test_create_pattern_intermediate_folders_firstc             C   s   | j   | jdd| j | jdd| j tj| jdd| jd dd	d
}tj| jd| jd d	d
}|d d }|d d }| jdd| j | jdd| jd d | jd| j | jd| j d S )Nr   z--encryption=repokeyrk  z--cache-onlyr   z--no-cache-syncz::testrN   z--jsonz--errorr  r   statsz::test2r   )r  r   r   r  r  )r   create_jsonZ	info_jsonZcreate_statsZ
info_statsrS   rS   rU   test_create_no_cache_sync  s    
z*ArchiverTestCase.test_create_no_cache_syncc             C   s   | j dd| j | jddd | jddd | jddd | j d	| jd
 d td | j ddd| jd
  W d Q R X | jttjddg d S )Nr   z--encryption=repokeyr   i   r   )r   r  r  r   z::testrN   r^   r   z--pattern=+input/file_importantz--pattern=-input/file*zoutput/inputi @ i @ i @ )r   r   r   r?   r   r  rb   r  )r   rS   rS   rU   test_extract_pattern_opt  s    
z)ArchiverTestCase.test_extract_pattern_optc             C   s\   t d | jd| jd  W d Q R X | jttjdddg | jttjdtg d S )Nr^   r   z::testzoutput/inputZcache2r   zoutput/input/cache2)r?   r   r   r   r  rb   r  r  )r   rS   rS   rU   _assert_test_caches  s    
z$ArchiverTestCase._assert_test_cachesc             C   s*   | j   | jdd| jd d | j  d S )Nr   z--exclude-cachesz::testrN   )r  r   r   r  )r   rS   rS   rU   test_exclude_caches  s    z$ArchiverTestCase.test_exclude_cachesc             C   s<   | j   | jd| jd d | jdd| jd  | j  d S )Nr   z::testrN   recreatez--exclude-caches)r  r   r   r  )r   rS   rS   rU   test_recreate_exclude_caches  s    z-ArchiverTestCase.test_recreate_exclude_cachesc             C   sD   | j dd| j | jdd
d | jd | jd | jd	dd d S )Nr   z--encryption=repokeyr   i   r   )r   ztagged1/.NOBACKUPztagged2/00-NOBACKUPztagged3/.NOBACKUP/file2i @ )r   r   r   )r   rS   rS   rU   _create_test_tagged  s
    

z$ArchiverTestCase._create_test_taggedc             C   sB   t d | jd| jd  W d Q R X | jttjddg d S )Nr^   r   z::testzoutput/inputr   )r?   r   r   r   r  rb   r  )r   rS   rS   rU   _assert_test_tagged  s    
z$ArchiverTestCase._assert_test_taggedc             C   s0   | j   | jddddd| jd d | j  d S )Nr   z--exclude-if-presentz	.NOBACKUPz00-NOBACKUPz::testrN   )r  r   r   r  )r   rS   rS   rU   test_exclude_tagged  s    z$ArchiverTestCase.test_exclude_taggedc             C   sB   | j   | jd| jd d | jddddd| jd  | j  d S )Nr   z::testrN   r  z--exclude-if-presentz	.NOBACKUPz00-NOBACKUP)r  r   r   r  )r   rS   rS   rU   test_recreate_exclude_tagged  s
    z-ArchiverTestCase.test_recreate_exclude_taggedc             C   s   | j dd| j | jddd | jd | jddd | jddd | jd	dd | jd
t td d | jddd | jd | jddd | jdt td d | jddd d S )Nr   z--encryption=repokeyfile0i   )r   ztagged1/.NOBACKUP1ztagged1/file1ztagged2/.NOBACKUP2/subfile1ztagged2/file2z
tagged3/%ss    extra stuff)r   ztagged3/file3ztaggedall/.NOBACKUP1ztaggedall/.NOBACKUP2/subfile1ztaggedall/%sztaggedall/file4)r   r   r   r  r  )r   rS   rS   rU   _create_test_keep_tagged  s    



z)ArchiverTestCase._create_test_keep_taggedc             C   s   t d | jd| jd  W d Q R X | jttjdddddd	g | jtjd
dg | jtjddg | jtjdtg | jttjdddtg d S )Nr^   r   z::testzoutput/inputr  Ztagged1Ztagged2Ztagged3Z	taggedallzoutput/input/tagged1z
.NOBACKUP1zoutput/input/tagged2z
.NOBACKUP2zoutput/input/tagged3zoutput/input/taggedall)r?   r   r   r   r  rb   r  r  )r   rS   rS   rU   _assert_test_keep_tagged  s    
 z)ArchiverTestCase._assert_test_keep_taggedc             C   s8   | j dd| j | j ddd| jd t}| jd| d S )Nr   z--encryption=repokeyr   z--exclude-cachesz--keep-tag-filesz::testz&--keep-tag-files" has been deprecated.)r   r   r   r  )r   output_warnrS   rS   rU   $test_exclude_keep_tagged_deprecation  s    z5ArchiverTestCase.test_exclude_keep_tagged_deprecationc          
   C   s4   | j   | jddddddd| jd d	 | j  d S )	Nr   z--exclude-if-presentz
.NOBACKUP1z
.NOBACKUP2z--exclude-cachesz--keep-exclude-tagsz::testrN   )r  r   r   r  )r   rS   rS   rU   test_exclude_keep_tagged  s    z)ArchiverTestCase.test_exclude_keep_taggedc          
   C   sF   | j   | jd| jd d | jddddddd	| jd  | j  d S )
Nr   z::testrN   r  z--exclude-if-presentz
.NOBACKUP1z
.NOBACKUP2z--exclude-cachesz--keep-exclude-tags)r  r   r   r  )r   rS   rS   rU   !test_recreate_exclude_keep_tagged  s
    z2ArchiverTestCase.test_recreate_exclude_keep_taggedc             C   s   | j dd| j | jdtd tjtjj| jd tj	tjj| jdtjj| jdt
 | j d| jd d | j d	d
d| jd  d S )Nr   z--encryption=noner   )r   Zsubdirr   z::testrN   r  z--exclude-cachesz--keep-exclude-tags)r   r   r   r  rb   r   r   r   r   r   r  )r   rS   rS   rU   test_recreate_hardlinked_tags  s    z.ArchiverTestCase.test_recreate_hardlinked_tagsz4Linux capabilities test, requires fakeroot >= 1.20.2c                s   t j  fdd}d}| jd tjdd| | jdd| j | jd	| jd
 d td2 tj	t d| | jd| jd
  W d Q R X W d Q R X d S )Nc                s"   t j| dd dd  | || d S )Nzsecurity.capabilityF)follow_symlinks)r   r   )r   Zuidgid)fchownrS   rU   patched_fchown'  s    zBArchiverTestCase.test_extract_capabilities.<locals>.patched_fchowns                     r3  z
input/filezsecurity.capabilityr   z--encryption=repokeyr   z::testrN   r^   r  r   )
rb   r  r   r   r   r   r   r?   r   object)r   r  ZcapabilitiesrS   )r  rU   test_extract_capabilities!  s    

z*ArchiverTestCase.test_extract_capabilitiesz@xattr not supported on this system or on this version offakerootc          !   C   s  dd }dd }dd }| j d tjdd	d
 | jd| jd | jd| jd d td tjjd}t	j
td| | jd| jd td}W d Q R X tj| t	j
td| | jd| jd td}W d Q R X tj| t	j
td| | jd| jd td}W d Q R X W d Q R X d S )Nc              _   s   t tjdd S )NE2BIG)r   r   r  )rT   kwargsrS   rS   rU   patched_setxattr_E2BIG9  s    zKArchiverTestCase.test_extract_xattrs_errors.<locals>.patched_setxattr_E2BIGc              _   s   t tjdd S )NENOTSUP)r   r   r  )rT   r  rS   rS   rU   patched_setxattr_ENOTSUP<  s    zMArchiverTestCase.test_extract_xattrs_errors.<locals>.patched_setxattr_ENOTSUPc              _   s   t tjdd S )Nr+  )r   r   r+  )rT   r  rS   rS   rU   patched_setxattr_EACCES?  s    zLArchiverTestCase.test_extract_xattrs_errors.<locals>.patched_setxattr_EACCESr3  z
input/filezuser.attributevaluer   z-enoner   z::testrN   r^   r   r   )rj   )r   r   r   r   r   r?   rb   r   abspathr   r  r+   r   )r   r  r  r  Zinput_abspathr   rS   rS   rU   test_extract_xattrs_errors6  s*    






z+ArchiverTestCase.test_extract_xattrs_errorsc             C   st   | j dd| j | jddd td | j d| jd	 d
 W d Q R X | j d| jd	 }| jd| | jd| d S )Nr   z--encryption=repokeyzdir1/dir2/filei   r   )r   zinput/dir1/dir2r   z::testz#../../../input/dir1/../dir1/dir2/..rl   z..z input/dir1/dir2/filei @ )r   r   r   r?   r  r  )r   r^   rS   rS   rU   test_path_normalizationU  s    
z(ArchiverTestCase.test_path_normalizationc          '   C   sT  | j dd| j | jddd | jddd td | j d	d
| jd d W d Q R X td | j d| jd  W d Q R X | jttjddg td | j d	d| jd d W d Q R X td | j d| jd  W d Q R X | jttjddg | j d	d| jd d td | j d| jd  W d Q R X | jttjddg d S )Nr   z--encryption=repokeyr   i   r   )r   r  rN   r   z--exclude=file1z::test1r  r^   r   z--exclude=./file1z::test2z--exclude=input/./file1z::test3zoutput/inputi @ i @ )r   r   r   r?   r   r  rb   r  )r   rS   rS   rU   test_exclude_normalization^  s"    
 

 

z+ArchiverTestCase.test_exclude_normalizationc             C   s8   | j dd
d | jdd| j | jd| jd d	d	 d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   i @ )r   r   r   )r   rS   rS   rU   test_repeated_filesq  s    z$ArchiverTestCase.test_repeated_filesc             C   s   | j ddd | j ddd | jdd| j | jd| jd	 d
 tjd tjd tjd td | jd| jd	  W d Q R X | jd
d tjd tjd tjd td | jd| jd	 dd W d Q R X d S )Nr   i   r   )r   z
dir2/file2r   z--encryption=repokeyr   z::testrN   zoutput/inputzoutput/input/file1zoutput/input/dir2r^   r   zoutput/input/file1/dirr;   )rj   i @ i @ )r   r   r   rb   r   r?   r  unlink)r   rS   rS   rU   test_overwritev  s    







zArchiverTestCase.test_overwritec          
   C   s:  | j ddd | j ddd | jdd| j | jd| jd	 d
 | jd| jd d
 | jdd| jd	  | jdd| jd  | jd| jd	 d | jdd| jd  | jd| jd d | jdd| jd  | jdd| jd  t| j}tj|tj\}}W d Q R X | jt	|j
d | jd|j
 | jd|j
 d S )Nr   i   r   )r   z
dir2/file2r   z--encryption=repokeyr   z::testrN   z::test.2r   z	--dry-runrh  ztest.3ztest.4z::test.3z::test.4r   i @ i @ )r   r   r   r:   r   r'   r   r   r   r\  archivesr  )r   r   r   rI   rS   rS   rU   test_rename  s"    zArchiverTestCase.test_renamec             C   sh   | j ddd | jdd| j | jd| jd d	 | jd
| j}| jd
| jd }| jd
dd| j}d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   r  z--firstry  i @ )r   r   r   )r   	info_repoinfo_archiverS   rS   rU   	test_info  s    zArchiverTestCase.test_infoc                s   | j ddd | jdd| j | jd| jd d	 tj| jd
d| j}|d }|d }|d  tj| jd
d| jd }|d }|d }d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   r  z--jsonr   r   r  r  r   i @ )r   r   r   r  r  )r   r  r   r   r  r  r   rS   )r  rU   test_info_json  s    zArchiverTestCase.test_info_jsonc             C   sD   | j dd| j tj| j ddd| j}tj| j ddd| j}dS )z2See https://github.com/borgbackup/borg/issues/6120r   z--encryption=repokeyr  z--jsonz	--first=1z--last=1N)r   r   r  r  )r   r  rS   rS   rU   test_info_json_of_empty_archive  s    z0ArchiverTestCase.test_info_json_of_empty_archivec             C   s   | j ddd | jdd| j | jd| jd d	 | jdd
d| jd d	 | jdd
d| jd d	 | jdd
d| jd d	 | jd| jd d
d | jd| jd d
d | jd| jd d
d | jd| jd d d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::test1rN   z	--commentzthis is the commentz::test2z"deleted" commentz::test3zpreserved commentz::test4r  zadded commentzmodified commentrQ   Z12345i @ )r   r   r   )r   rS   rS   rU   test_comment  s    zArchiverTestCase.test_commentc          
   C   sL  | j ddd | j ddd | jdd| j | jd| jd	 d
 | jd| jd d
 | jd| jd d
 | jd| jd d
 | jd| jd d
 | jdd| jd	  | jdd| jd  | jddd| j | jddd| j | jd| jd	  | jdd| jd  | jdd| jd }| jd| t| j}| jt|d W d Q R X d S )Nr   i   r   )r   z
dir2/file2r   z--encryption=repokeyr   z::testrN   z::test.2z::test.3z::another_test.1z::another_test.2r   z	--dry-runrk  z--prefixZanother_z--lastry  z--statszDeleted data:r;   i @ i @ )r   r   r   r  r:   r   r   r\  )r   r^   r   rS   rS   rU   test_delete  s$    zArchiverTestCase.test_deletec             C   s   | j ddd | jdd| j | jd| jd d	 | jd| jd
 d	 | jd| jd d	 | jd| jd d | jdd| jd  | jd| jd d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::test1rN   z::test2z::test3rk  test2r   z	--dry-runZtest3i @ )r   r   r   )r   rS   rS   rU   test_delete_multiple  s    z%ArchiverTestCase.test_delete_multiplec             C   s   | j ddd | j ddd | jdd| j | jd| jd	 d
 | jd| jd d
 dtjd< | jd| jdd dtjd< | jd| j | jtjj| j d S )Nr   i   r   )r   z
dir2/file2r   z--encryption=repokeyr   z::testrN   z::test.2nor   rk  r   )rj   r   i @ i @ )	r   r   r   rb   r   ZassertFalser   r   r   )r   rS   rS   rU   test_delete_repo  s    

z!ArchiverTestCase.test_delete_repoc             C   s   | j dd| j | jd t| jdd`}tj|tj\}}t|||d}x0|j	 D ]$}|j
jdrT|j|jd j P qTW |j  W d Q R X | j dd	| jd
 }| jd| | j dd| j | j d| j}| jd| d S )Nr   z--encryption=noner
  T)r   ztestsuite/archiver.pyr;   rk  z--forcez::testzdeleted archive was corruptedr   z--repairrl   )r   r   r   r:   r   r'   r   r   r   
iter_itemsr   endswithrk  chunksrI  commitr  r  )r   r   r   rI   r   r  r^   rS   rS   rU   test_delete_force  s    
z"ArchiverTestCase.test_delete_forcec             C   s   | j dd| j | jd t| jddF}tj|tj\}}t|||d}|j	j
d }|j|d |j  W d Q R X | j dd	d	| jd
  | j dd| j | j d| j}| jd| d S )Nr   z--encryption=noner
  T)r   r   s%   corrupted items metadata stream chunkrk  z--forcez::testr   z--repairrl   )r   r   r   r:   r   r'   r   r   r   metadataitemsputr  r  )r   r   r   rI   r   rI  r^   rS   rS   rU   test_delete_double_force"  s    
z)ArchiverTestCase.test_delete_double_forcec             C   s   | j dd| j | jd | j dd| jd  | j dd| j}| jd	| | jd
| ttjtjj	| j
dddddd }ttjj	| j
ddd|d}|jd |jd W d Q R X | j dd| jdd}| jd
| d S )Nr   z--encryption=repokeyr
  r   z	--dry-runz::testr   z--show-versionzborgbackup versionzStarting repository checkr   r   rz  T)reverser;   zr+br   s   XXXXz--info)rj   )r   r   r   r  r  r  rb   r  r   r   r   r   rU  r   )r   r^   r   r   rS   rS   rU   test_corrupted_repository0  s    
&
z*ArchiverTestCase.test_corrupted_repositoryc             C   s   | j dd| j | jd | j| jn | jrD| j dd| jtd n:tjt	t
jf}| j dd| j W d Q R X t|jt
jr~| j dd| jd W d Q R X d S )Nr   z--encryption=repokeyr
  r   z--verify-data)rj   z--bypass-lock)r   r   r   	read_onlyr   r   r,   r%  r&  r6   r8   RPCErrorrY   r  )r   excinforS   rS   rU   test_readonly_check?  s    
z$ArchiverTestCase.test_readonly_checkc             C   s   | j dd| j | jd | jd | j| jz | jrR| j dd| j dtd n>tjt	t
jf}| j dd| j d W d Q R X t|jt
jr| j dd| j dd W d Q R X d S )	Nr   z--encryption=repokeyr  r  diffz%s::a)rj   z--bypass-lock)r   r   r   r  r   r   r,   r%  r&  r6   r8   r  rY   r  )r   r  rS   rS   rU   test_readonly_diffN  s    

z#ArchiverTestCase.test_readonly_diffc             C   s   | j dd| j | jd | j| jz | jrH| j dd| j dtd n>tjt	t
jf}| j dd| j d W d Q R X t|jt
jr| j dd| j dd W d Q R X d S )	Nr   z--encryption=repokeyr
  z
export-tarz%s::testztest.tar)rj   z--bypass-lock)r   r   r   r  r   r   r,   r%  r&  r6   r8   r  rY   r  )r   r  rS   rS   rU   test_readonly_export_tar^  s    
z)ArchiverTestCase.test_readonly_export_tarc             C   s   | j dd| j | jd | j| jt | jrF| j dd| j td n<tjt	t
jf}| j dd| j  W d Q R X t|jt
jr| j dd| j d W d Q R X d S )Nr   z--encryption=repokeyr
  r   z%s::test)rj   z--bypass-lock)r   r   r   r  r   r   r,   r%  r&  r6   r8   r  rY   r  )r   r  rS   rS   rU   test_readonly_extractm  s    
z&ArchiverTestCase.test_readonly_extractc             C   s   | j dd| j | jd | j| jh | jrB| j d| jtd n8tjt	t
jf}| j d| j W d Q R X t|jt
jrz| j d| jd W d Q R X d S )Nr   z--encryption=repokeyr
  r  )rj   z--bypass-lock)r   r   r   r  r   r   r,   r%  r&  r6   r8   r  rY   r  )r   r  rS   rS   rU   test_readonly_info|  s    
z#ArchiverTestCase.test_readonly_infoc             C   s   | j dd| j | jd | j| jh | jrB| j d| jtd n8tjt	t
jf}| j d| j W d Q R X t|jt
jrz| j d| jd W d Q R X d S )Nr   z--encryption=repokeyr
  rl   )rj   z--bypass-lock)r   r   r   r  r   r   r,   r%  r&  r6   r8   r  rY   r  )r   r  rS   rS   rU   test_readonly_list  s    
z#ArchiverTestCase.test_readonly_listc             C   s   | j dd| j | jd | j| j | jrL| j| jtd W d Q R X nFtj	t
tjf"}| j| jdd W d Q R X W d Q R X t|jtjr| j| jd d W d Q R X W d Q R X d S )Nr   z--encryption=repokeyr
  )rj   F)rL   z--bypass-lock)r   r   r   r  r   r   r  r,   r%  r&  r6   r8   r  rY   r  )r   r  rS   rS   rU   test_readonly_mount  s    
z$ArchiverTestCase.test_readonly_mountBORG_TESTS_IGNORE_MODESzmodes unreliablec             C   sV   | j ddd | jdd| j | jd| jd d	 tj| jj}| jtj|d
 d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   i  i @ )	r   r   r   rb   r   r   st_modeassertEqualS_IMODE)r   moderS   rS   rU   
test_umask  s
    zArchiverTestCase.test_umaskc          
   C   sd   | j dd| j | j dd| jd d t| j}tj|tj\}}W d Q R X | jt|j	d d S )Nr   z--encryption=repokeyr   z	--dry-runz::testrN   r   )
r   r   r:   r   r'   r   r   r   r\  r  )r   r   r   rI   rS   rS   rU   test_create_dry_run  s
    z$ArchiverTestCase.test_create_dry_runc          
   C   sZ   t | jddB}tj|tj\}}|jj ddgii|jd< |j  |j	  W d Q R X d S )NT)r   s	   mandatorys   unknown-features   feature_flags)
r:   r   r'   r   r   r  r   rK  r   r  )r   Z	operationr   r   rI   rS   rS   rU   add_unknown_feature  s
    z$ArchiverTestCase.add_unknown_featurec          	   C   s<   | j r| j|dti n tjt}| j|  W d Q R X d S )Nrj   )r   r   r,   r%  r&  r(   )r   rT   r  rS   rS   rU   cmd_raises_unknown_feature  s
    z+ArchiverTestCase.cmd_raises_unknown_featurec             C   s<   t | jdd| j | jtjj | jd| jd dg d S )Nr   z--encryption=repokeyr   z::testrN   )r   r   r   r  r'   	OperationZWRITEr  )r   rS   rS   rU   test_unknown_feature_on_create  s    z/ArchiverTestCase.test_unknown_feature_on_createc             C   sH   | j dd| j | j dd| j | jtjj | jd| jd dg d S )Nr   z--encryption=repokeyrk  z--cache-onlyr   z::testrN   )r   r   r  r'   r  READr  )r   rS   rS   rU   "test_unknown_feature_on_cache_sync  s    z3ArchiverTestCase.test_unknown_feature_on_cache_syncc             C   s6   t | jdd| j | jtjj | jd| jg d S )Nr   z--encryption=repokeyzchange-passphrase)r   r   r   r  r'   r  CHECKr  )r   rS   rS   rU   )test_unknown_feature_on_change_passphrase  s    z:ArchiverTestCase.test_unknown_feature_on_change_passphrasec             C   s   t | jdd| j | jd| jd d | jtjj td | jd| jd g W d Q R X | jd| jg | jd	| jd g d S )
Nr   z--encryption=repokeyr   z::testrN   r^   r   rl   r  )	r   r   r   r  r'   r  r  r?   r  )r   rS   rS   rU   test_unknown_feature_on_read  s    
z-ArchiverTestCase.test_unknown_feature_on_readc             C   sP   t | jdd| j | jd| jd d | jtjj | jd| jd dg d S )Nr   z--encryption=repokeyr   z::testrN   rh  other)r   r   r   r  r'   r  r  r  )r   rS   rS   rU   test_unknown_feature_on_rename  s    z/ArchiverTestCase.test_unknown_feature_on_renamec             C   sn   t | jdd| j | jd| jd d | jtjj | jd| jd g | jdd| jg | jd| j d S )	Nr   z--encryption=repokeyr   z::testrN   rk  prunez--keep-daily=3)r   r   r   r  r'   r  ZDELETEr  )r   rS   rS   rU   test_unknown_feature_on_delete  s    z/ArchiverTestCase.test_unknown_feature_on_deletec             C   sf   | j dd| j | j d| jd d | jtjj tjj| j	d}tj
| | jd| jd |g d S )Nr   z--encryption=repokeyr   z::testrN   r  r   )r   r   r  r'   r  r  rb   r   r   r   r   r  )r   r  rS   rS   rU   test_unknown_feature_on_mount  s    
z.ArchiverTestCase.test_unknown_feature_on_mountc          %      sR  | j rd}nd}t| jdd| j t| jdd^}|rDt| j|_tj	|tj
\}}t|||$}|j  tdg|j_|j  W d Q R X W d Q R X | jr| jd| jd	 d
 nFd tj fdd}tjtd| | jd| jd	 d
 W d Q R X t| jddB}|rt| j|_tj	|tj
\}}t|||}W d Q R X W d Q R X d S )Nzssh://__testsuite__rQ   r   z--encryption=repokeyT)r   zunknown-featurer   z::testrN   Fc                 s   d |   d S )NTrS   )rT   )calledwipe_cache_saferS   rU   wipe_wrapper  s    zNArchiverTestCase.test_unknown_mandatory_feature_in_cache.<locals>.wipe_wrapper
wipe_cache)r   r   r   r   r:   r   r%   Z	_locationr'   r   r   r   	begin_txnrM  Zcache_configZmandatory_featuresr  r   r   r  r   r  )r   path_prefixr   r   rI   r   r  rS   )r  r  rU   'test_unknown_mandatory_feature_in_cache  s2    z8ArchiverTestCase.test_unknown_mandatory_feature_in_cachec             C   sD   | j ddd | jdd| j | jdd| jd	 d
}| jd| d S )Nr   i   r   )r   r   z--encryption=repokeyr   z
--progressz::test4rN   i @ )r   r   r   r  )r   r^   rS   rS   rU   test_progress_on*  s    z!ArchiverTestCase.test_progress_onc             C   sB   | j ddd | jdd| j | jd| jd d	}| jd
| d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::test5rN   r  i @ )r   r   r   r  )r   r^   rS   rS   rU   test_progress_off0  s    z"ArchiverTestCase.test_progress_offc             C   s   | j ddd tjd | j ddd | jdd| j | jd	d
| jd d}| jd| | jd| | jd	d
| jd d}| jd| | jd| dS )ztest that various file status show expected results

        clearly incomplete: only tests for the weird "unchanged" status for nowr   i   r   )r   r;   r  r   z--encryption=repokeyr   z--listz::testrN   zA input/file1zA input/file2z::test1zU input/file1Ni @ i @ )r   r   r  r   r   r  )r   r^   rS   rS   rU   test_file_status6  s    
z!ArchiverTestCase.test_file_statusc             C   s   | j ddd tjd | j ddd | jdd	| j | jd
dd| jd d}tjd}| j ddd tjd|j|j	fd | jd
dd| jd d}| j
d| dS )zbtest that a changed file with faked "previous" mtime still gets backed up in ctime,size cache_moder   s   123)r   r;   r  r   )r   r   z--encryption=repokeyr   z--listz--files-cache=ctime,sizez::test1rN   zinput/file1s   321)nsz::test2zM input/file1N)r   r   r  r   r   rb   r   r9  r/  st_mtime_nsr  )r   r^   rX  rS   rS   rU   test_file_status_cs_cache_modeH  s    

z/ArchiverTestCase.test_file_status_cs_cache_modec             C   s   | j ddd tjd | j ddd | jdd| j | jdd	d
| jd d}tjd}tjd|jtj	A  | jdd	d
| jd d}| j
d| dS )zetest that a chmod'ed file with no content changes does not get chunked again in mtime,size cache_moder   r   )r   r;   r  r   z--encryption=repokeyr   z--listz--files-cache=mtime,sizez::test1rN   zinput/file1z::test2zU input/file1N)r   r   r  r   r   rb   r   r   r  S_IRWXOr  )r   r^   rX  rS   rS   rU   test_file_status_ms_cache_modeW  s    

z/ArchiverTestCase.test_file_status_ms_cache_modec             C   sv   | j ddd tjd | j ddd | jdd| j | jdd	d
| jd d}| jdd	d
| jd d}| jd| dS )zItest that files get rechunked unconditionally in rechunk,ctime cache moder   r   )r   r;   r  r   z--encryption=repokeyr   z--listz--files-cache=rechunk,ctimez::test1rN   z::test2zA input/file1N)r   r   r  r   r   r  )r   r^   rS   rS   rU   test_file_status_rc_cache_modee  s    
z/ArchiverTestCase.test_file_status_rc_cache_modec             C   s   | j ddd tjd | j ddd trR| j ddd tjtjj| j	dt
j | jdd	| j | jd
dd| jd d}| jd| | jd| tr| jd| | jd
dd| jd ddd}| jd| | jd| tr| jd| dS )z#test that excluded paths are listedr   i   r   )r   r;   r  r  r   z--encryption=repokeyr   z--listz--exclude-nodumpz::testrN   zA input/file1zA input/file2zx input/file3z::test1z	--excludez*/file2zU input/file1zx input/file2Ni @ i @ i @ )r   r   r  r<   r   r   rb   r   r   r   r   r   r   r   r  )r   r^   rS   rS   rU   test_file_status_excludedp  s"    
z*ArchiverTestCase.test_file_status_excludedc             C   sF   | j ddd | jdd| j tj| jdd| jd	 d
}|d }d S )Nr   i   r   )r   r   z--encryption=repokeyr   z--jsonz::testrN   r   i @ )r   r   r   r  r  )r   Zcreate_infor   rS   rS   rU   test_create_json  s
    z!ArchiverTestCase.test_create_jsonc             C   s   | j ddd tjd | j ddd | jdd| j | jd	| jd
 d}| jd| | jd	| jd d}| jd| | jd	dd| jd d}| jd| | jd	dd| jd d}| jd| | j ddd | jd	dd| jd d}| jd| d S )Nr   i   r   )r   r;   r  r   z--encryption=repokeyr   z::testrN   z::test0z--listz
--filter=Uz::test1z--filter=AMz::test2r   z::test3i @ i @ i  )r   r   r  r   r   r  r  )r   r^   rS   rS   rU   test_create_topical  s    
z$ArchiverTestCase.test_create_topicalc             C   sR   t jdt jj| jd | jdd| j | jd }| jdd|d | jd	|}d S )
Nzsomewhere does not existr   r   z--encryption=repokeyz::testr   z--read-specialrN   rl   )rb   r   r   r   r   r   r   )r   r   r^   rS   rS   rU   'test_create_read_special_broken_symlink  s    
z8ArchiverTestCase.test_create_read_special_broken_symlinkc             C   s  | j dd| j | j d| jd t | j d| jd t | j d| jd t | j d| jd t | j d| jd t | j d	d
d| jd}| jd| | jd| | jd| | jd| | j d| j}| jd| | jd| | jd| | jd| | jd| | j d	| jd | j d| j}| jd| | jd| | jd| | jd| | jd| | j d| jd t | j d	| jd | j d| j}| jd| | jd| d S )Nr   z--encryption=repokeyr   z::test1z::test2z::test3.checkpointz::test3.checkpoint.1z::test4.checkpointr  z--listz	--dry-runz--keep-daily=2zKeeping archive: test2zWould prune:     test1z!Keeping archive: test4.checkpointrl   test1r  ztest3.checkpointztest3.checkpoint.1ztest4.checkpointz::test5Z
checkpointZtest5)r   r   r   r  r  )r   r^   rS   rS   rU   test_prune_repository  s:    z&ArchiverTestCase.test_prune_repositoryc             C   s   | j dd| j | j d| jd t | j d| jd t | j ddd| jd	}| jd
| | jd| | j d| j}| jd| | jd| | j dd| jd	 | j d| j}| jd| | jd| d S )Nr   z--encryption=repokeyr   z::test1z::test2r  z--listz	--dry-runz--keep-daily=2zKeeping archive: test2zWould prune:     test1rl   r  r  z--save-space)r   r   r   r  r  )r   r^   rS   rS   rU    test_prune_repository_save_space  s    z1ArchiverTestCase.test_prune_repository_save_spacec             C   s   | j dd| j | j d| jd t | j d| jd t | j d| jd t | j d| jd t | j dd	d
| jdd}| jd| | jd| | j d| j}| jd| | jd| | jd| | jd| | j d| jdd | j d| j}| jd| | jd| | jd| | jd| d S )Nr   z--encryption=repokeyr   z::foo-2015-08-12-10:00z::foo-2015-08-12-20:00z::bar-2015-08-12-10:00z::bar-2015-08-12-20:00r  z--listz	--dry-runz--keep-daily=2z--prefix=foo-z%Keeping archive: foo-2015-08-12-20:00z%Would prune:     foo-2015-08-12-10:00rl   zfoo-2015-08-12-10:00zfoo-2015-08-12-20:00zbar-2015-08-12-10:00zbar-2015-08-12-20:00)r   r   r   r  r  )r   r^   rS   rS   rU   test_prune_repository_prefix  s&    z-ArchiverTestCase.test_prune_repository_prefixc             C   s   | j dd| j | j d| jd t | j d| jd t | j d| jd t | j d| jd t | j dd	d
| jdd}| jd| | jd| | j d| j}| jd| | jd| | jd| | jd| | j d| jdd | j d| j}| jd| | jd| | jd| | jd| d S )Nr   z--encryption=repokeyr   z::2015-08-12-10:00-fooz::2015-08-12-20:00-fooz::2015-08-12-10:00-barz::2015-08-12-20:00-barr  z--listz	--dry-runz--keep-daily=2z--glob-archives=2015-*-fooz%Keeping archive: 2015-08-12-20:00-fooz%Would prune:     2015-08-12-10:00-foorl   z2015-08-12-10:00-fooz2015-08-12-20:00-fooz2015-08-12-10:00-barz2015-08-12-20:00-bar)r   r   r   r  r  )r   r^   rS   rS   rU   test_prune_repository_glob  s&    z+ArchiverTestCase.test_prune_repository_globc             C   s   | j dd| j | j d| jd t | j d| jd t | j d| jd t | j dd| j}| jd	| | jd
| | jd| d S )Nr   z--encryption=repokeyr   z::test-1z::something-else-than-test-1z::test-2rl   z--prefix=test-ztest-1ztest-2zsomething-else)r   r   r   r  r  )r   r^   rS   rS   rU   test_list_prefix  s    z!ArchiverTestCase.test_list_prefixc             C   s   | j dd| j | jd }| j d|t | j ddd|}| jd| | j d|}| j dd	d
|}| j dd	d|}| j|| | j|| d S )Nr   z--encryption=repokeyz::testr   rl   z--list-formatr  z#--list-format" has been deprecated.z--formatzB{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NEWLINE}z{mtime:%s} {path}{NL})r   r   r   r  r  ZassertNotEqual)r   test_archiver  output_1output_2output_3rS   rS   rU   test_list_format'  s    
z!ArchiverTestCase.test_list_formatc             C   s   | j dd| j | j ddd| jd t | j ddd| jd t | j d	| j}| j d	d
d| j}| j|| | j d	d| j}| j|d | j d	d
d| j}| j|d | j d	d
d| j}| jd| | jd| d S )Nr   z--encryption=repokeyr   z	--commentz	comment 1z::test-1z	comment 2z::test-2rl   z--formatz{archive:<36} {time} [{id}]{NL}z--shortztest-1
test-2
z{barchive}/ztest-1/test-2/z{name} {comment}{NL}ztest-1 comment 1
ztest-2 comment 2
)r   r   r   r  r  )r   r!  r"  r#  rS   rS   rU   test_list_repository_format3  s    z,ArchiverTestCase.test_list_repository_formatc             C   s\   | j ddd | j ddd d | jdd	| j | jd
 }| jd|d | jddd|}d S )N
empty_filer   )r   Zamb   ai@B )r   r   z--encryption=repokeyz::testr   rN   rl   z--formatz{sha256} {path}{NL})r   r   r   )r   r   r^   rS   rS   rU   test_list_hashB  s    
zArchiverTestCase.test_list_hashc             C   s   | j ddd | j d ttjj| jdd"}|jdd  |jdd  W d Q R X | jd	d
| j | jd }| jd|d | jddd|}d S )Nr&  r   )r   Z
two_chunksr   s   abbai s   baabr   z--encryption=repokeyz::testr   rN   rl   z--formatz'{num_chunks} {unique_chunks} {path}{NL})	r   r   rb   r   r   r   r   r   r   )r   r   r   r^   rS   rS   rU   test_list_chunk_countsL  s    

z'ArchiverTestCase.test_list_chunk_countsc             C   sl   | j ddd | jdd| j | jd }| jddd	|d
 | jddd|}|jdd jd\}}}}}d S )NZcompressible_filei'  )r   r   z--encryption=repokeyz::testr   z-Clz4rN   rl   z--formatz*{size} {csize} {dsize} {dcsize} {path}{NL}r  r;    )r   r   r   r  )r   r   r^   r   csizeZdsizeZdcsizer   rS   rS   rU   test_list_sizeY  s    
zArchiverTestCase.test_list_sizec             C   s   | j ddd | jdd| j | jd| jd d	 tj| jd
d| j}|d }|d d }| jd
d| jd }dd |j D }|d }| jd
dd| jd }dd |j D }|d }d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   rl   z--jsonr   r  r   z--json-linesc             S   s   g | ]}t j|qS rS   )r  r  )r   srS   rS   rU   r  s  s    z3ArchiverTestCase.test_list_json.<locals>.<listcomp>r;   z--format={sha256}c             S   s   g | ]}t j|qS rS   )r  r  )r   r.  rS   rS   rU   r  {  s    i @ )r   r   r   r  r  r   )r   Z	list_repor   Zarchive0Zlist_archiver  r   rS   rS   rU   test_list_jsone  s    zArchiverTestCase.test_list_jsonc             C   s@   | j dd| j | j dd| jdd | j dd| jd dd d S )	Nr   z--encryption=repokeyrl   z--json-linesr   )rj   z--jsonz	::archive)r   r   )r   rS   rS   rU   test_list_json_args  s    z$ArchiverTestCase.test_list_json_argsc             C   sr   | j   | jdd| j | jdd| jd ddd}i }x&|j D ]}tj|}|||d	 < q@W |d
 }|d }d S )Nr   z--encryption=repokeyr   z
--log-jsonz::testrN   z--listz--debugtypefile_statuslog_message)r  r   r   r   r  r  )r   logZmessagesr   msgr2  r3  rS   rS   rU   test_log_json  s    
zArchiverTestCase.test_log_jsonc             C   s   | j   | jdd| j | jd| jd dd | jddd	d
 tjd
}|j  |jd | jd| jd dd tjd
}|j  |jd d S )Nr   z--encryption=repokeyr   z::testrN   z--debug-profile=create.profdebugzconvert-profilezcreate.profzcreate.pyprofZcumtimez::test2z--debug-profile=create.pyprof)r  r   r   pstatsZStatsZ
strip_dirsZ
sort_stats)r   r  rS   rS   rU   test_debug_profile  s    


z#ArchiverTestCase.test_debug_profilec             C   s2   | j   | jdd| j | jdd| jd d}d S )Nr   z--encryption=repokeyz--debugr   z::testrN   )r  r   r   )r   r4  rS   rS   rU   test_common_options  s    z$ArchiverTestCase.test_common_options'  c       	      C   s   |rd| }n
t j|}| jd|d | jdd| j | jd }| jdd||d	 | jd
dd|}|jdd jd\}}}t|t|fS )Nr   r3  )r   r   z--encryption=nonez::testr   z-CrN   rl   z--formatz{size} {csize} {path}{NL}r  r;   r+  )rb   r   r   r   r   r  int)	r   Zcompressioncompressibler   r   r   r^   r,  r   rS   rS   rU   
_get_sizes  s    


zArchiverTestCase._get_sizesc             C   s   | j ddd\}}d S )NnoneT)r=  )r>  )r   r   r,  rS   rS   rU   "test_compression_none_compressible  s    z3ArchiverTestCase.test_compression_none_compressiblec             C   s   | j ddd\}}d S )Nr?  F)r=  )r>  )r   r   r,  rS   rS   rU   $test_compression_none_uncompressible  s    z5ArchiverTestCase.test_compression_none_uncompressiblec             C   s   | j ddd\}}d S )NzlibT)r=  )r>  )r   r   r,  rS   rS   rU   "test_compression_zlib_compressible  s    z3ArchiverTestCase.test_compression_zlib_compressiblec             C   s   | j ddd\}}d S )NrB  F)r=  )r>  )r   r   r,  rS   rS   rU   $test_compression_zlib_uncompressible  s    z5ArchiverTestCase.test_compression_zlib_uncompressiblec             C   s   | j ddd\}}d S )Nr*  T)r=  )r>  )r   r   r,  rS   rS   rU   !test_compression_lz4_compressible  s    z2ArchiverTestCase.test_compression_lz4_compressiblec             C   s   | j ddd\}}d S )Nr*  F)r=  )r>  )r   r   r,  rS   rS   rU   #test_compression_lz4_uncompressible  s    z4ArchiverTestCase.test_compression_lz4_uncompressiblec             C   s   | j ddd\}}d S )NlzmaT)r=  )r>  )r   r   r,  rS   rS   rU   "test_compression_lzma_compressible  s    z3ArchiverTestCase.test_compression_lzma_compressiblec             C   s   | j ddd\}}d S )NrG  F)r=  )r>  )r   r   r,  rS   rS   rU   $test_compression_lzma_uncompressible  s    z5ArchiverTestCase.test_compression_lzma_uncompressiblec             C   s`   | j ddd\}}| jd| j | j ddd\}}| jd| j | j ddd\}}t||}d S )Nz	auto,zlibT)r=  rk  rB  r*  )r>  r   r   min)r   Z	auto_sizeZ
auto_csizeZ	zlib_sizeZ
zlib_csizeZlz4_sizeZ	lz4_csizeZsmallest_csizerS   rS   rU   "test_compression_auto_compressible  s    
z3ArchiverTestCase.test_compression_auto_compressiblec             C   s   | j ddd\}}d S )Nz	auto,zlibF)r=  )r>  )r   r   r,  rS   rS   rU   $test_compression_auto_uncompressible  s    z5ArchiverTestCase.test_compression_auto_uncompressiblec             C   sD   | j dd| j dtjd< | j d| j dtjd< | j d| j d S )Nr   z--encryption=repokeyZnewpassphraseZBORG_NEW_PASSPHRASEzchange-passphraser   rl   )r   r   rb   r   )r   rS   rS   rU   test_change_passphrase  s
    

z'ArchiverTestCase.test_change_passphrasec             C   s"   | j dd| j | j d| j d S )Nr   z--encryption=repokeyz
break-lock)r   r   )r   rS   rS   rU   test_break_lock  s    z ArchiverTestCase.test_break_lockc             C   s   | j   | j d d S )Nz-h)r   )r   rS   rS   rU   
test_usage  s    zArchiverTestCase.test_usagec             C   s   d S )NrS   )r   rS   rS   rU   	test_help	  s    zArchiverTestCase.test_helpc          2   C   s  dd }| j dd| j | j  |d}| j dd| jd d	 | j dd| jd
 d	 trltjtjjd	d tjj| jd}| j	| j|F | j
| jtjj|dd	ddd | j
| jtjj|dd	ddd W d Q R X | j	| jd | | j
| jtjj|d	ddd d}tjj|d	d}tj|}tj|}|r8t r@t|d}t|d}	W d Q R X W d Q R X t rd}tjj|d	d}tj|}
tj|}t rd}tjj|d	d}tj|dd}tj|dd}t rtjj|d	d}tj|}yhd}tjj|d	d}tj r&tj| jr&n6ytj|d W n$ tk
rZ } zW Y d d }~X nX W nD tk
r } z&tjjdr|jtjkrn W Y d d }~X nX W d Q R X d S )Nc             S   s\   t j| j}yt jt j| t W n tk
r6   dS X t j| j}ttk}|oV||kS d S )NF)rb   r   r/  r,  r   r   r   r   )r1  r2  r4  r5  rS   rS   rU   r6  
	  s    z/ArchiverTestCase.test_fuse.<locals>.has_noatimer   z--encryption=repokeyzinput/file1r   z--exclude-nodumpz	::archiverN   z
::archive2r   r  r   T)ignore_bsdflagsignore_xattrsarchive2r   rR  zinput/hardlinkr   zinput/link1r   F)r  r   zinput/fusexattrr   zuser.foofreebsd)rT  )r   r   r  r<   rb   r   r   r   r   r  r  r   r   rC   r   rB   rD   r   r   r   getxattrr   rW   r   r  r   r  )r   r6  r<  r  Zin_fnZout_fnZsti1Zsto1Zin_fZout_fZsti2Zsto2r=  r>  rr   r   rS   rS   rU   	test_fuse	  sh    






zArchiverTestCase.test_fusec                sV  | j dd| j | jddd t rJ| jddd tjdd	 tjdd
 | j d| jd d | jddd | j d| jd d tjj| jd}| j	| j|dd\ tjj|dd tj
 }t rtjj|ddd}tjj|ddd}tjj|ddd}W d Q R X | j	| j|dddd2 t rHtjj|ddd}tjj|ddd}W d Q R X d S )Nr   z--encryption=repokeyr
  s   first)r   Z	hardlink1s   123456zinput/hardlink1zinput/hardlink2zinput/hardlink3r   z
::archive1rN   s   secondz
::archive2r  z-oZversionszhardlink1.00001Z	hardlink2zhardlink2.00001Z	hardlink3zhardlink3.00001z-e)r   r   r   rC   rb   r   r   r   r   r  r  )r   r  filesZhl1Zhl2Zhl3rS   )r   rU   test_fuse_versions_viewn	  s.    

z(ArchiverTestCase.test_fuse_versions_viewc              C   s  | j dd| j | jd | jd\}}|F x6|j D ]*}|jjdr8|j|jd j	 |j}P q8W |j
  W d Q R X | j dd| jdd	 tjj| jd
}| j| jd |. tjt}ttjj|| W d Q R X W d Q R X | j| jd |dd ttjj||j  W d Q R X d S )Nr   z--encryption=repokeyr   ztestsuite/archiver.pyr;   r   z--repairr   )rj   r  z	::archivez-oZallow_damaged_filesr  )r   r   r   r   r  r   r  rk  r  rI  r  rb   r   r   r  r%  r&  r   r   r,  )r   r   r   r  r   r  r  rS   rS   rU   test_fuse_allow_damaged_files	  s$    

z.ArchiverTestCase.test_fuse_allow_damaged_filesc          ,   C   s   | j dd| j | jd | jd | jd | jd tjj| jd}| j| j|dd	 W d Q R X | j| j|d
d	 W d Q R X | j| j|d W d Q R X | j| j|d W d Q R X | j| j|d W d Q R X | j| j|d W d Q R X d S )Nr   z--encryption=repokeyZarch11Zarch12Zarch21Zarch22r  z	--first=2z--sort=namez--last=2z--prefix=arch1z--prefix=arch2z--prefix=archz--prefix=nope)r   r   r   rb   r   r   r   r  )r   r  rS   rS   rU   test_fuse_mount_options	  s$    








z(ArchiverTestCase.test_fuse_mount_optionsc                s   t   t   fdd}j  dtjd< jdd| j |  jdjd d	 |  jdjd
 d	 |  jdjd
  |  d S )Nc                 s   t j} x| j| j j D ]~\}}| j|}t|j }| kr  j| t	t
|d }t|dd }x,t||| D ]}j| j| qW q W W d Q R X d S )N)   !   )r:   r   Z
open_indexZget_transaction_id	iteritemsr   r   Zdigestaddr   r\  r   r   r  )r   rI  _r   hashZ
num_blocksnonceZcounter)seenr   usedrS   rU   verify_uniqueness	  s    

zIArchiverTestCase.verify_aes_counter_uniqueness.<locals>.verify_uniquenessrc  r   r   z--encryption=r   z::testrN   z::test.2rk  )rM  r  rb   r   r   r   )r   methodrd  rS   )rb  r   rc  rU   verify_aes_counter_uniqueness	  s    
z.ArchiverTestCase.verify_aes_counter_uniquenessc             C   s   | j d d S )Nkeyfile)rf  )r   rS   rS   rU   #test_aes_counter_uniqueness_keyfile	  s    z4ArchiverTestCase.test_aes_counter_uniqueness_keyfilec             C   s   | j d d S )Nrepokey)rf  )r   rS   rS   rU   &test_aes_counter_uniqueness_passphrase	  s    z7ArchiverTestCase.test_aes_counter_uniqueness_passphrasec             C   sf   | j   | jdd| j | jd| jd d td | jdd| jd }W d Q R X ttjd}d S )	Nr   z--encryption=repokeyr   z::testrN   r^   r7  zdump-archive-items)r  r   r   r?   r  rb   r  )r   r^   
output_dirrS   rS   rU   test_debug_dump_archive_items	  s    
z.ArchiverTestCase.test_debug_dump_archive_itemsc             C   sb   | j   | jdd| j | jd| jd d td | jdd| j}W d Q R X ttjd}d S )	Nr   z--encryption=repokeyr   z::testrN   r^   r7  zdump-repo-objs)r  r   r   r?   r  rb   r  )r   r^   rk  rS   rS   rU   test_debug_dump_repo_objs	  s    
z*ArchiverTestCase.test_debug_dump_repo_objsc             C   s   | j dd| j d}t|j }| jd|d | j dd| jd}| j dd	| j|d
}td
d}|j }W d Q R X | j dd| j|}| j dd| j|}| j dd| jd}d S )Nr   z--encryption=repokeys	   some datar3  )r   r7  zput-objz
input/filezget-objzoutput/filerR  z
delete-objZinvalid)r   r   r   Z	hexdigestr   r   r0  )r   r   Zhexkeyr^   r   Z	data_readrS   rS   rU   test_debug_put_get_delete_obj	  s    z.ArchiverTestCase.test_debug_put_get_delete_objc             C   s:   dd }t jtd| | jdd| jdd W d Q R X d S )Nc              W   s   t d S )N)EOFError)rT   rS   rS   rU   	raise_eof
  s    z7ArchiverTestCase.test_init_interrupt.<locals>.raise_eofr   r   z--encryption=repokeyr;   )rj   )r   r  r   r   r   )r   rp  rS   rS   rU   test_init_interrupt
  s    z$ArchiverTestCase.test_init_interruptc             C   s   | j d| jdd d S )Nr   r   )rj   )r   r   )r   rS   rS   rU   $test_init_requires_encryption_option
  s    z5ArchiverTestCase.test_init_requires_encryption_optionc             C   s`   | j dd| j | jr0| j dd| jd dd n,tjtj | j dd| jd  W d Q R X d S )Nr   z--encryption=repokeyz/nestedr   )rj   )r   r   r   r%  r&  r:   ZAlreadyExists)r   rS   rS   rU   test_init_nested_repositories
  s
    z.ArchiverTestCase.test_init_nested_repositoriesc             C   s   t jj| jd}t|d | jdd| jd  t|}|j }W dQ R X dd| jd f}| j	rt| j|dd	i n$t
jtjj | j|  W dQ R X t|}|j }W dQ R X W dQ R X dS )
zBORG_KEY_FILE=something borg init should quit if "something" already exists.

        See https://github.com/borgbackup/borg/pull/6046rg  )ZBORG_KEY_FILEr   z--encryption=keyfilerz  Nry  rj   r   )rb   r   r   r   r@   r   r   r   r0  r   r%  r&  rp   r   Error)r   rg  r3  beforeargafterrS   rS   rU   %test_init_refuse_to_overwrite_keyfile
  s    

z6ArchiverTestCase.test_init_refuse_to_overwrite_keyfilec             C   s   | j d| j | j b}tj|tj\}}t|||dd}|j}W d Q R X tj| t|||}|j}W d Q R X W d Q R X t	 }x2|j
 D ]&\}\}	}
}|| \}}}|j| qW x|j
 D ]\}\}	}
}qW d S )Nr   F)sync)r   r   r   r'   r   r   r   r  ZdestroyrM  r]  r^  )r   r   r   rI   r   Zoriginal_chunksZcorrect_chunksrb  rI  Zrefcountr   r,  Z
o_refcountZo_sizeZo_csizerS   rS   rU   check_cache,
  s    

zArchiverTestCase.check_cachec             C   s   | j dd| j | j d| jd d | j b}tj|tj\}}t|||dd4}|j  |jj	t
|jj d d  |j  W d Q R X W d Q R X tjt | j  W d Q R X d S )	Nr   z--encryption=repokeyr   z::testrN   F)ry  r   )r   r   r   r'   r   r   r   r	  r  Zincrefrl   r]  r  r%  r&  AssertionErrorrz  )r   r   r   rI   r   rS   rS   rU   test_check_cacheB
  s    
z!ArchiverTestCase.test_check_cachec             C   s(   | j dd| j | j d| jddd}d S )Nr   z--encryption=repokeyr  z--target=asdfr   )rj   )r   r   )r   r^   rS   rS   rU   test_recreate_target_rcN
  s    z(ArchiverTestCase.test_recreate_target_rcc             C   s   | j   | jdd| j | j  | jd }| jd|d | j  | jd| j}| jd|dd	d
d | j  | jd| j}| jd }| jdd|}d S )Nr   z--encryption=repokeyz::test0r   rN   rl   r  z
input/dir2z-ezinput/dir2/file3z--target=new-archivez::new-archivez--short)r  r   r   rz  )r   r   Zoriginal_archiver  listingrS   rS   rU   test_recreate_targetS
  s    

z%ArchiverTestCase.test_recreate_targetc             C   sj   | j   | jddd | jdd| j | jd }| jd|d	 | jd
|ddd | j  | jdd|}d S )Nz
dir2/file3i   r   )r   r   z--encryption=repokeyz::test0r   rN   r  z
input/dir2z-ezinput/dir2/file3rl   z--shorti @ )r  r   r   r   rz  )r   r   r~  rS   rS   rU   test_recreate_basicg
  s    
z$ArchiverTestCase.test_recreate_basicc             C   s   | j   | jd| jd d | jd| jd d | j  td | jd| jd  W d Q R X td | jd| jd  W d Q R X d S )	Nr   z::test2rN   r  z::testz
input/dir1r^   r   )r~  r   r   rz  r?   )r   rS   rS   rU   test_recreate_subtree_hardlinkst
  s    


z0ArchiverTestCase.test_recreate_subtree_hardlinksc             C   s   t tjj| jdd"}|jdd  |jdd  W d Q R X | jdd| j | jdd	d
| jd d | jd| jd dd | jd| jd ddd}tt	|j
d\}}| jd| jd	d | j  d S )NZ
large_filer   r'  i     br   z--encryption=repokeyr   z--chunker-paramsz	7,9,8,128z::test1rN   z::test2z--files-cache=disabledrl   zinput/large_filez--formatz{num_chunks} {unique_chunks}r+  r  default)r   rb   r   r   r   r   r   r   mapr<  r  rz  )r   r   rl   Z
num_chunksZunique_chunksrS   rS   rU   test_recreate_rechunkify
  s    z)ArchiverTestCase.test_recreate_rechunkifyc             C   s   | j ddd | jdd| j | jd| jd dd	d
 | jd| jd ddd}|jd\}}}| jd| jd	dd | j  | jd| jd ddd}|jd\}}}d S )Nr=  i'  )r   r   z--encryption=repokeyr   z::testrN   z-Cr?  rl   zinput/compressiblez--formatz{size} {csize} {sha256}r+  r  r*  z--recompress)r   r   r   r  rz  )r   Z	file_listr   r,  Zsha256_beforeZsha256_afterrS   rS   rU   test_recreate_recompress
  s    z)ArchiverTestCase.test_recreate_recompressc                s   t jttdj j}| j  | jdd| j | jd }| jd|d | jddd	d
d| | jd|j	 }t ddd|j
d  }|jd d S )Nr   r   z--encryption=repokeyz::test0r   rN   r  z--timestampz1970-01-02T00:00:00z	--commentr
  r  i  r;   r   z%Y-%m-%d)r   nowr   r   Z
astimezoneZtzinfor  r   r   r   Z	utcoffsetstrftime)r   Zlocal_timezoner   r  ZdtimerS   )Zs_timerU   test_recreate_timestamp
  s    

z(ArchiverTestCase.test_recreate_timestampc             C   sv   | j ddd | jdd| j | jd| jd d | jd	| jd }| jd
| jddd | j  | jd	| jd }d S )Nr=  i'  )r   r   z--encryption=repokeyr   z::testrN   rl   r  z-nz-ezinput/compressible)r   r   r   rz  )r   Zarchives_beforeZarchives_afterrS   rS   rU   test_recreate_dry_run
  s    z&ArchiverTestCase.test_recreate_dry_runc             C   st   | j ddd | jdd| j | jd| jd d	 | jd
| jd }| jd| jdd | j  | jd
| jd }d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   r  r  z--chunker-paramsr  i @ )r   r   r   rz  )r   Zinfo_beforeZ
info_afterrS   rS   rU   !test_recreate_skips_nothing_to_do
  s    z2ArchiverTestCase.test_recreate_skips_nothing_to_doc             C   sN   | j dd| j tjj| jd}ddd| f}| j d| jf|dd	d
 d S )Nr   z--encryption=repokeyzlock.exclusiveZpython3z-cz<import os, sys; sys.exit(42 if os.path.exists("%s") else 23)z	with-lockT*   )rL   rj   )r   r   rb   r   r   r   )r   Z	lock_pathr   rS   rS   rU   test_with_lock
  s    zArchiverTestCase.test_with_lockc             C   sN  | j dd| j | jddd | jddd | jddd | jddd | jd	dd | j d
| jd d | j ddd| jd dd}| j  | jd| | jd| | j dd| jd dd}| j  | jd| | jd| | j d| jd dd}| j  | jd| | jd| | j dd| jd dd}| j  | jd| | jd| d S )Nr   z--encryption=repokeyr   r   )r   r  r  r  Zfile5r   z::testrN   r  z--listz--infoz-ezinput/file2zinput/file1zx input/file2zinput/file3zx input/file3zinput/file4zx input/file4zinput/file5zx input/file5)r   r   r   rz  r  r  )r   r^   rS   rS   rU   test_recreate_list_output
  s.    z*ArchiverTestCase.test_recreate_list_outputc          
   C   sD   | j dd| j | j d| jd d | j dddd	d| jd
dd d S )Nr   z--encryption=repokeyr   z::testrN   rk  z--firstry  z--lastTr   )rL   rj   )r   r   )r   rS   rS   rU   test_bad_filters
  s    z!ArchiverTestCase.test_bad_filtersc             C   s   | j d }| jd| jdd | j| j}| jdd| j| t|d}|j }W d Q R X | jd tj	| jd	  }t|d}|j }W d Q R X tj
| | jdd
| j| t|d}|j }W d Q R X d S )Nz	/exportedr   z--encryptionrg  rI   exportr/r   import)r   r   r   r  r   r   r0  r   rb   r  r  )r   export_filerepo_idr   export_contentskey_fileZkey_contentsZkey_contents2rS   rS   rU   test_key_export_keyfile
  s    

z(ArchiverTestCase.test_key_export_keyfilec       	      C   s  | j d }| jd| jdd | j| j}| jdd| j| t|d}|j }W d Q R X t| j}t|}|j	d t
j  W d Q R X ttjj }|j	|t
j  t| j}|jd W d Q R X | jdd	| j| t| j}t|}|j	d t
j  W d Q R X d S )
Nz	/exportedr   z--encryptionri  rI   r  r  rJ   r  )r   r   r   r  r   r   r0  r:   r   r   r   Zenv_passphraser   rI   ZTestKeyZMockRepositoryZsave_key)	r   r  r  r   r  r   Zrepo_keyZ
backup_keyZ	repo_key2rS   rS   rU   test_key_export_repokey  s$    
z(ArchiverTestCase.test_key_export_repokeyc          	   C   sb   | j d }| jd| jdd | j| j}| jddd| j| t|dd	d
}|j }W d Q R X d S )Nz/exported.htmlr   z--encryptionri  rI   r  z	--qr-htmlr  zutf-8)rR   )r   r   r   r  r   r   r0  )r   r  r  r   r  rS   rS   rU   test_key_export_qr-  s    
z#ArchiverTestCase.test_key_export_qrc             C   s@   | j d }tj| | jd| jdd | jdd| j|td d S )Nz	/exportedr   z--encryptionri  rI   r  )rj   )r   rb   r   r   r   r,   )r   Zexport_directoryrS   rS   rU   test_key_export_directory:  s    

z*ArchiverTestCase.test_key_export_directoryc          !   C   s  | j d }| jd| jdd | jdd| j|td t|d}|jd	 W d Q R X | jrp| jdd| j|d
d n(tjt	 | jdd| j| W d Q R X t|d}|jd W d Q R X | jr| jdd| j|d
d n(tjt
 | jdd| j| W d Q R X d S )Nz	/exportedr   z--encryptionrg  rI   r  )rj   rL  zsomething not a key
r   zBORG_KEY a0a0a0
)r   r   r   r,   r   r   r   r%  r&  r#   r"   )r   r  r   rS   rS   rU   test_key_import_errorsB  s    
z'ArchiverTestCase.test_key_import_errorsc             C   s   d}| j d }| jd| jdd | j| jt| | jd tj| jd  }t	|d0}|j
tjd	 | d
  |j
tdj  W d Q R X | jddd| j| t	|d}|j }W d Q R X d S )N@e294423506da4e1ea76e8dcdf1a3919624ae3ae496fddf905610c351d3f09239z	/exportedr   z--encryptionrg  r  r   rL  r+  r  s   abcdefghijklmnopqrsturI   r  z--paperr  )r   r   r   rN  r   r   r   rb   r  r   r   r   FILE_IDr   rf   r0  )r   r  r  r  r   r  rS   rS   rU   test_key_export_paperkeyZ  s    
z)ArchiverTestCase.test_key_export_paperkeyc             C   s   d}| j d| jdd | j| jt| | jd tj| jd  }t|d0}|j	t
jd | d	  |j	td
j  W d Q R X d}| j ddd| j|d d}| j ddd| j|d d}| j ddd| j|d d S )Nr  r   z--encryptionrg  r  r   rL  r+  r  s   abcdefghijklmnopqrstus  2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41  02
2 / e29442 3506da 4e1ea7  25f62a 5a3d41 - 02
2 / e29442 3506da 4e1ea7 / 25f62a 5a3d42 - 02
2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02
616263 646566 676869 6a6b6c 6d6e6f 707172 - 6d


737475 88
73747i - 88
73747 - 88
73 74 75  -  89
00a1 - 88
2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02
616263 646566 676869 6a6b6c 6d6e6f 707172 - 6d
73 74 75  -  88
rI   r  z--paper)rN   s   
y
s1   2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02

y
)r   r   rN  r   r   r   rb   r  r   r   r   r  r   rf   )r   r  r  r   Ztyped_inputrS   rS   rU   test_key_import_paperkeyt  s    z)ArchiverTestCase.test_key_import_paperkeyc          	   C   sr   | j ddd | jdd| j | jd| jd d	 | jd
 }| jdd| j|}t|d}tj|}W d Q R X d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   z/dumpr7  zdump-manifestr  i @ )r   r   r   r   r   r  r   )r   	dump_filer^   r   r  rS   rS   rU   test_debug_dump_manifest  s    
z)ArchiverTestCase.test_debug_dump_manifestc          	   C   sv   | j ddd | jdd| j | jd| jd d	 | jd
 }| jdd| jd |}t|d}tj|}W d Q R X d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   z/dumpr7  zdump-archiver  i @ )r   r   r   r   r   r  r   )r   r  r^   r   r  rS   rS   rU   test_debug_dump_archive  s    
z(ArchiverTestCase.test_debug_dump_archivec             C   s   | j dd| j | j dd| jdd j }tj| j dd| jd	 d
}|d d }| j dd| j|j }| j dd| jddj }d S )Nr   z--encryption=repokeyr7  zrefcount-objrz  @   r   z--jsonz::testrN   r   rI  Z124Zxyza)r   r   stripr  r  )r   r^   r  
archive_idrS   rS   rU   test_debug_refcount_obj  s    z(ArchiverTestCase.test_debug_refcount_objc             C   s   | j dd}d S )Nr7  r  )r   )r   r^   rS   rS   rU   test_debug_info  s    z ArchiverTestCase.test_debug_infoc             C   s>   | j dd| j tdd | j dd| j| j W d Q R X d S )Nr   z--encryption=repokeyr   )Z_BORG_BENCHMARK_CRUD_TESTZ	benchmarkZcrud)r   r   r@   r   )r   rS   rS   rU   test_benchmark_crud  s    z$ArchiverTestCase.test_benchmark_crudc             C   s0  | j   tjd | jdd| j | jdd| j}| jd| | jd| | jd| | jd	| | jd
| | jd| | jd| xhdD ]`\}}| jd| j|}| jd| j|| | jd| j|}| jdd| j| | jd| j|dd qW | jddd| jdd | jd| jdd | jd| jddd d S )Nzinput/flagfiler   z--encryption=repokeyrK  z--listz[repository]versionZsegments_per_dirZstorage_quotaappend_onlyadditional_free_spacerI  2Grepository.append_onlyry  z--deleter;   )rj   r   zinvalid-optionr  r  r  ry  )r  r  )r  rb   r  r   r   r  )r   r^   Zcfg_keyZ	cfg_valuerS   rS   rU   test_config  s,    
 zArchiverTestCase.test_configz(GNU tar must be installed for this test.gzipz%gzip must be installed for this test.c             C   s   | j   tjd | jdd| j | jd| jd d | jd| jd dd	 td
 tjddddg W d Q R X | jdddddd d S )Nzinput/flagfiler   z--encryption=repokeyr   z::testrN   z
export-tarz
simple.tarz
--progressr^   rt   xpfz../simple.tarz--warning=no-timestampzoutput/inputT)rQ  rR  	ignore_ns)	r  rb   r  r   r   r?   r]   
check_callr  )r   rS   rS   rU   test_export_tar  s    

z ArchiverTestCase.test_export_tarc             C   s   t jdstjd | j  tjd | jdd| j | jd| jd d | jd	| jd d
d}t	d t
jddddg W d Q R X | jdddddd d S )Nr  zgzip is not installedzinput/flagfiler   z--encryption=repokeyr   z::testrN   z
export-tarzsimple.tar.gzz--listr^   rt   r  z../simple.tar.gzz--warning=no-timestampzoutput/inputT)rQ  rR  r  )ru   rv   r%  r*  r  rb   r  r   r   r?   r]   r  r  )r   rl   rS   rS   rU   test_export_tar_gz   s    



z#ArchiverTestCase.test_export_tar_gzc             C   s   t jdstjd | j  tjd | jdd| j | jd| jd d | jd	| jd d
dd}t	d t
jddddg W d Q R X | jdddddd d S )Nr  zgzip is not installedzinput/flagfiler   z--encryption=repokeyr   z::testrN   z
export-tarz
simple.tarz--strip-components=1z--listr^   rt   r  z../simple.tarz--warning=no-timestampzoutput/T)rQ  rR  r  )ru   rv   r%  r*  r  rb   r  r   r   r?   r]   r  r  )r   rl   rS   rS   rU    test_export_tar_strip_components  s    



z1ArchiverTestCase.test_export_tar_strip_componentsc             C   sH   | j   | jd| jd dd td tjdddd	g W d Q R X d S )
Nz
export-tarz::testz
output.tarz--strip-components=2r^   rt   r  z../output.tarz--warning=no-timestamp)r~  r   r   r?   r]   r  )r   rS   rS   rU   &test_export_tar_strip_components_links   s
    
z7ArchiverTestCase.test_export_tar_strip_components_linksc             C   sH   | j   | jd| jd dd td tjdddd	g W d Q R X d S )
Nz
export-tarz::testz
output.tarz
input/dir1r^   rt   r  z../output.tarz--warning=no-timestamp)r~  r   r   r?   r]   r  )r   rS   rS   rU   test_extract_hardlinks_tar,  s
    
z+ArchiverTestCase.test_extract_hardlinks_tarc             C   s   t | j}d|d | jgd|d gd|gd|d dgd|gd|gd	|gd
|d gdd|dgdd|dgd|gd|gg}x|D ]}| j|ddd}qzW d S )Nr   z::testr   r   rh  Znewnamerl   rk  r  r  rI   r  Zexportedr  zchange-passphrasez
break-lockTr   )rL   rj   )rH   r   r   r   )r   r   ZcmdsrT   r^   rS   rS   rU   test_detect_attic_repo8  s     






z'ArchiverTestCase.test_detect_attic_repoc             C   s   dd }| j d tjddd | jd| jd | jd	| jd
 d td6 tjtd| | jd| jd
 td W dQ R X W dQ R X dS )z.https://github.com/borgbackup/borg/issues/6063c              _   s   t tjdd S )Nr+  )r   r   r+  )rT   r  rS   rS   rU   r  Q  s    z`ArchiverTestCase.test_do_not_fail_when_percent_is_in_xattr_name.<locals>.patched_setxattr_EACCESr3  s
   input/files   user.attribute%ps   valuer   z-enoner   z::testrN   r^   r   r   )rj   N)	r   r   r   r   r   r?   r   r  r+   )r   r  rS   rS   rU   .test_do_not_fail_when_percent_is_in_xattr_nameM  s    

z?ArchiverTestCase.test_do_not_fail_when_percent_is_in_xattr_namec             C   s   dd }t jt jj| jd tjddd | jd| jd | jd	| jd
 d t	d6 t
jtd| | jd| jd
 td W dQ R X W dQ R X dS )z.https://github.com/borgbackup/borg/issues/6063c              _   s   t tjdd S )Nr+  )r   r   r+  )rT   r  rS   rS   rU   r  a  s    z_ArchiverTestCase.test_do_not_fail_when_percent_is_in_file_name.<locals>.patched_setxattr_EACCESzdir%ps   input/dir%ps   user.attributes   valuer   z-enoner   z::testrN   r^   r   r   )rj   N)rb   r   r   r   r   r   r   r   r   r?   r   r  r+   )r   r  rS   rS   rU   -test_do_not_fail_when_percent_is_in_file_name]  s    
z>ArchiverTestCase.test_do_not_fail_when_percent_is_in_file_namec             C   s   | j ddd | jdd| j | jd| jd d tjtjj| jd	 tjtjj| j d	 | jd
| j}| jd| j}| jd
| jd }| jd| jd }t	d | jd| jd  W dQ R X | j
dd dS )zNonce is only used for encrypting new data.

        It should be possible to retrieve the data from an archive even if
        both the client and the server forget the noncer   s   Hello, borg)r   r   z--encryption=repokeyr   z::testrN   ra  r  rl   r^   r   Nzoutput/input)r   r   r   rb   r   r   r   r   r&   r?   r  )r   Z	repo_infoZ	repo_listZarchive_infor"  rS   rS   rU   +test_can_read_repo_even_if_nonce_is_deletedl  s    
z<ArchiverTestCase.test_can_read_repo_even_if_nonce_is_deletedc             C   sd   | j ddd | jdd| j | jd| jd d tjj| jd	}tj| | jd| jd
 d dS )a   We should be able to recover if path/to/repo/nonce is deleted.

        The nonce is stored in two places: in the repo and in $HOME.
        The nonce in the repo is only needed when multiple clients use the same
        repo. Otherwise we can just use our own copy of the nonce.
        r   s   Hello, borg)r   r   z--encryption=repokeyr   z::testrN   ra  z::test2N)r   r   r   rb   r   r   r   r   )r   ra  rS   rS   rU   %test_recovery_from_deleted_repo_nonce  s    
z6ArchiverTestCase.test_recovery_from_deleted_repo_nonceN)r;  )r  r  r  r%  markskipifrC   Zrequires_hardlinksr&   r  r$  r(  r-  rB   r.  rE   r?  rF   rD  rH  r  rN  r`  rb  re  rj  rl  rm  ro  rt  ru  rv  r|  r~  unittest
skipUnlessr=   r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r   r   r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  rb   r   r  r  r  r  r  r  r  r  r  r  r  Zallow_cache_wiper  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r$  r%  r(  r)  r-  r/  r0  r6  r9  r:  r>  r@  rA  rC  rD  rE  rF  rH  rI  rK  rL  rM  rN  rO  rP  rV  rX  rY  rZ  rf  rh  rj  rl  rm  rn  rq  rr  rs  rx  rz  r|  r}  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  rz   Zrequires_gnutarru   rv   Zrequires_gzipr  r  r  r  r  r  r  r  r  r  rS   rS   rS   rU   r    s  ?		3!!(!		
	


)'

f 

 

-  r  zno borg.exe availablec                   s   e Zd ZdZdZejddd Zejddd Zejdd	d
 Z	ejddd Z
ejddd Zejddd Z fddZejddd Zejddd Z  ZS )ArchiverTestCaseBinaryzborg.exeTz(does not raise Exception, but sets rc==2c             C   s   d S )NrS   )r   rS   rS   rU   r(    s    z,ArchiverTestCaseBinary.test_init_parent_dirszpatches objectsc             C   s   d S )NrS   )r   rS   rS   rU   rq    s    z*ArchiverTestCaseBinary.test_init_interruptc             C   s   d S )NrS   )r   rS   rS   rU   r    s    z0ArchiverTestCaseBinary.test_extract_capabilitiesc             C   s   d S )NrS   )r   rS   rS   rU   r    s    z1ArchiverTestCaseBinary.test_extract_xattrs_errorszLtest_basic_functionality seems incompatible with fakeroot and/or the binary.c             C   s   d S )NrS   )r   rS   rS   rU   r    s    z/ArchiverTestCaseBinary.test_basic_functionalityzBtest_overwrite seems incompatible with fakeroot and/or the binary.c             C   s   d S )NrS   )r   rS   rS   rU   r    s    z%ArchiverTestCaseBinary.test_overwritec                s    t  rtjd n
t j  d S )Nz9test_fuse with the binary is not compatible with fakeroot)rG   r  r*  superrV  )r   )	__class__rS   rU   rV    s    z ArchiverTestCaseBinary.test_fusec             C   s   d S )NrS   )r   rS   rS   rU   r    s    zEArchiverTestCaseBinary.test_do_not_fail_when_percent_is_in_xattr_namec             C   s   d S )NrS   )r   rS   rS   rU   r    s    zDArchiverTestCaseBinary.test_do_not_fail_when_percent_is_in_file_name)r  r  r  r   r   r  r*  r(  rq  r  r  r  r  rV  r  r  __classcell__rS   rS   )r  rU   r    s   r  c                   s   e Zd Z fddZdd Zdd Zdd Zd	d
 Zdd Zdd Z	dd Z
dd Zdd Zdd Zdd Zdd Zdd Zdd Z  ZS )ArchiverCheckTestCasec                sL   t  j  tjtdd* | jdd| j | jd | jd W d Q R X d S )NBUFFER_SIZEr   r   z--encryption=repokeyarchive1rS  )r  r   r   r  r   r   r   r   )r   )r  rS   rU   r     s
    

zArchiverCheckTestCase.setUpc             C   s<  | j ddd| jdd}| jd| | jd| | jd| tjd	jtj | j ddd
| jdd}| jd| | jd| | jd| | j ddd| jdd}| jd| | jd| | j dddd| jdd}| jd| | j dddd| jdd}| jd| | jd| | j dddd| jdd}| jd| | jd| d S )Nr   z-vz
--progressr   )rj   zStarting repository checkz"Starting archive consistency checkzChecking segmentszborg.output.progressz--repository-onlyz--archives-onlyz--prefix=archive2r  z	--first=1rS  z--last=1)r   r   r  loggingZ	getLoggerZsetLevelZNOTSETr  )r   r^   rS   rS   rU   test_check_usage  s(    z&ArchiverCheckTestCase.test_check_usagec             C   s,  | j d\}}|R xB|j D ],}|jjdr|j}|d }|j|j P qW | jd |j  W d Q R X | j	d| j
dd | j	dd| j
dd}| jd	| | j	d| j
dd | j	d
d| j
d dd}| jd| xtdD ]l}| j |\}}|P xH|j D ]2}|jjd r| j||j | j||j P  qW | jd W d Q R X qW tjtdd | jd W d Q R X | j	ddd| j
dd}| jd| | jd| xhdD ]`}| j |\}}|B x:|j D ]$}|jjdr| j||j P qW | jd W d Q R X qW | j	d
d| j
d dd}| jd| d S )Nr  ztestsuite/archiver.pyr;   zshould not happenr   )rj   z--repairr   zNew missing file chunk detectedrl   z--format={health}#{path}{LF}z
::archive1zbroken#rS  r  r   Zarchive3z-vz$Healed previously missing file chunkzAtestsuite/archiver.py: Completely healed previously damaged file!r  )r  rS  )r  rS  )r   r  r   r  r  rk  rI  Zfailr  r   r   r  Zassert_not_equalr  r   r  r   r   r   )r   r   r   r  Zvalid_chunksZkilled_chunkr^   archive_namerS   rS   rU   test_missing_file_chunk  sP    


z-ArchiverCheckTestCase.test_missing_file_chunkc             C   st   | j d\}}|  |j|jjd  |j  W d Q R X | jd| jdd | jdd| jdd | jd| jdd d S )Nr  r   r   r;   )rj   z--repair)r   rk  r  r  r  r   r   )r   r   r   rS   rS   rU   test_missing_archive_item_chunk  s    z5ArchiverCheckTestCase.test_missing_archive_item_chunkc             C   sn   | j d\}}| |j|j |j  W d Q R X | jd| jdd | jdd| jdd | jd| jdd d S )Nr  r   r;   )rj   z--repairr   )r   rk  rI  r  r   r   )r   r   r   rS   rS   rU   test_missing_archive_metadata#  s    z3ArchiverCheckTestCase.test_missing_archive_metadatac             C   s   | j d\}}| |jtj |j  W d Q R X | jd| jdd | jddd| jdd}| jd| | jd| | jd| jdd d S )	Nr  r   r;   )rj   z-vz--repairr   rS  )r   rk  r'   MANIFEST_IDr  r   r   r  )r   r   r   r^   rS   rS   rU   test_missing_manifest,  s    z+ArchiverCheckTestCase.test_missing_manifestc             C   s   | j d\}}|0 |jtj}|d }|jtj| |j  W d Q R X | jd| jdd | jddd| jdd}| jd| | jd	| | jd| jdd d S )
Nr  s
   corrupted!r   r;   )rj   z-vz--repairr   rS  )	r   r   r'   r  r  r  r   r   r  )r   r   r   r   corrupted_manifestr^   rS   rS   rU   test_corrupted_manifest7  s    z-ArchiverCheckTestCase.test_corrupted_manifestc             C   s   | j d\}}|R |jtj}|d }|jtj| |j|j}|d }|j|j| |j  W d Q R X | jd| jdd | jddd| jdd}| j	d	| | jd| jdd d S )
Nr  s
   corrupted!r   r;   )rj   z-vz--repairr   rS  )
r   r   r'   r  r  rI  r  r   r   r  )r   r   r   r   r  chunkZcorrupted_chunkr^   rS   rS   rU   %test_manifest_rebuild_corrupted_chunkD  s    z;ArchiverCheckTestCase.test_manifest_rebuild_corrupted_chunkc       	      C   s   | j d\}}|j}|n |jtj}|d }|jtj| g g dddddd}|j|dd	}|j|}|j||j| |j	  W d Q R X | j
d
| jdd | j
d
d| jdd | j
d| j}| jd| | jd| | jd| d S )Nr  s
   corrupted!foobarz2016-12-15T18:49:51.849711r;   )cmdliner  hostnameusernamer   r   r  s   archive)contextr   )rj   z--repairr   rl   z
archive1.1rS  )r   rI   r   r'   r  r  Zpack_and_authenticate_metadataid_hashencryptr  r   r   r  )	r   r   r   rI   r   r  Zarchive_dictr  r^   rS   rS   rU   'test_manifest_rebuild_duplicate_archiveT  s.    
z=ArchiverCheckTestCase.test_manifest_rebuild_duplicate_archivec             C   s   | j d| jdd t| jdd}|jdd |j  W d Q R X | j d| jdd | j d| jdd | j dd	| jdd | j d| jdd | j d
d| jd dd d S )Nr   r   )rj   T)r   s    01234567890123456789012345678901s   xxxxr;   z--repairr   z	--dry-runz
::archive1)r   r   r:   r  r  )r   r   rS   rS   rU   test_extra_chunksp  s    z'ArchiverCheckTestCase.test_extra_chunksc             G   s   t j| j | jd| jf|  | jd | jd\}}|V xF|j D ]:}|jj	drH|j
d }|j|jd }|j|j| P qHW |j  W d Q R X | jd| jdd | jdd	| jdd}| jdd
d	| jdd}d S )Nr   r  ztestsuite/archiver.pyr;   s   1234r   r   )rj   z--verify-dataz--repairr  )ru   r   r   r   r   r   r   r  r   r  r  r   rI  r  r  )r   Z	init_argsr   r   r  r  r   r^   rS   rS   rU   _test_verify_data{  s     

z'ArchiverCheckTestCase._test_verify_datac             C   s   | j dd d S )Nz--encryptionri  )r  )r   rS   rS   rU   test_verify_data  s    z&ArchiverCheckTestCase.test_verify_datac             C   s   | j dd d S )Nz--encryptionr?  )r  )r   rS   rS   rU   test_verify_data_unencrypted  s    z2ArchiverCheckTestCase.test_verify_data_unencryptedc             C   sT   t | jdd*}x|j D ]}|j| qW |j  W d Q R X | jd| jdd d S )NT)r   r   r;   )rj   )r:   r   rl   rk  r  r   )r   r   Zid_rS   rS   rU   test_empty_repository  s
    z+ArchiverCheckTestCase.test_empty_repositoryc             C   s   G dd d}| j d\}}|Z tj|tj\}}t|||0}t|||d|dd}|jj|  |j  W d Q R X W d Q R X | j	d| j
dd	 | j	d
| j
d dd	 d S )Nc               @   s   e Zd Zdd ZdS )zAArchiverCheckTestCase.test_attic013_acl_bug.<locals>.Attic013Itemc          	   S   s   dddddddd dS )NZ1234r      0)s   paths   mtimes   modes   users   groups   uids   gids   aclrS   )r   rS   rS   rU   as_dict  s    zIArchiverCheckTestCase.test_attic013_acl_bug.<locals>.Attic013Item.as_dictN)r  r  r  r  rS   rS   rS   rU   Attic013Item  s   r  r  z0.13T)r   r   r   r   )rj   rl   z::0.13)r   r'   r   r   r   r   Zitems_bufferr^  Zsaver   r   )r   r  r   r   r   rI   r   rS   rS   rU   test_attic013_acl_bug  s    z+ArchiverCheckTestCase.test_attic013_acl_bug)r  r  r  r   r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  rS   rS   )r  rU   r    s   2		r  c               @   s4   e Zd Zdd Zdd Zdd Zdd Zd	d
 ZdS )ManifestAuthenticationTestc             C   sf   |X t j|t j\}}|jt j|jtjdi i tj	 t
dd jtd |j  W d Q R X d S )Nr;   )days)r  r  rK  	timestamp)r'   r   r   r  r  r  r/   packbr   utcnowr   r  
ISO_FORMATr  )r   r   r_  rI   rS   rS   rU   spoof_manifest  s    "z)ManifestAuthenticationTest.spoof_manifestc             C   s   | j dd| j t| jdd}|V tj|tj\}}|jtj|j	t
jdi tj tdd jtd |j  W d Q R X tjt | j d| j W d Q R X d S )	Nr   z--encryption=repokeyT)r   r;   )r  )r  r  r  rl   )r   r   r:   r   r'   r   r   r  r  r  r/   r  r   r  r   r  r  r  r%  r&  r    )r   r   r   rI   rS   rS   rU   test_fresh_init_tam_required  s    "z7ManifestAuthenticationTest.test_fresh_init_tam_requiredc             C   s4  | j dd| j | jd t| jdd}| tjtt|j	 t
j|t
j\}}d|_|j|j tj|jd |jt
j}|d= |jt
j|jtj| |j  W d Q R X | j dd	| j}| j d
d| j | j dd	| j}| j| tjt | j d| j W d Q R X | j d
dd| j | j d| j d S )Nr   z--encryption=repokeyarchive1234T)r   Fs   tamrl   z--debugupgradez--tamz--force)r   r   r   r:   r   ru   r   r&   r-   rI  r'   r   r   Ztam_requiredZchange_passphraseZ_passphraser/   ZunpackbZdecryptr   r  r  r  r  r  r  r%  r&  r    )r   r   r_  rI   r   r^   rS   rS   rU   test_not_required  s(    

z,ManifestAuthenticationTest.test_not_requiredc             C   sF   | j dd| j | jd | j dd| j t| jdd}| j| d S )Nr   z--encryption=repokeyr  r  z--disable-tamT)r   )r   r   r   r:   r   r  )r   r   rS   rS   rU   test_disable  s    

z'ManifestAuthenticationTest.test_disablec             C   sF   | j dd| j | jd t| jdd}| j| | j dd| j d S )Nr   z--encryption=repokeyr  T)r   r  z--disable-tam)r   r   r   r:   r   r  )r   r   rS   rS   rU   test_disable2  s    

z(ManifestAuthenticationTest.test_disable2N)r  r  r  r  r  r  r  r  rS   rS   rS   rU   r    s
    r  c               @   s,   e Zd Zdd Zdd Zdd Zdd Zd	S )
ArchiveAuthenticationTestc          	   C   sx   t j|t j\}}tjd|g dddtj jtd}|j	|}|j
||j| |tj f|j|< |j  |j  d S )Nr;   rQ   )r  r   r  r  r  r  r   )r'   r   r   r/   r  r   r  r  r  r  r  r  r  r  r   r  )r   r   r  r   rI   Zarchive_datar  rS   rS   rU   write_archive_without_tam	  s    
z3ArchiveAuthenticationTest.write_archive_without_tamc             C   sx   | j dd| j | jd t| jdd}| | j|d W d Q R X | j dd| j}| j d	d
| j | j dd| j}d S )Nr   z--encryption=repokeyarchive_tamT)r   archive_no_tamrl   z--format="{name} tam:{tam}{NL}"r  z--archives-tam)r   r   r   r:   r   r  )r   r   r^   rS   rS   rU   test_upgrade_archives_tam  s    
z3ArchiveAuthenticationTest.test_upgrade_archives_tamc             C   s|   | j dd| j | jd t| jdd}|& | j|d |jtj |j	  W d Q R X | j dd| j | j d	d
| j}d S )Nr   z--encryption=repokeyr  T)r   r  r   z--repairrl   z--format="{name} tam:{tam}{NL}")
r   r   r   r:   r   r  rk  r'   r  r  )r   r   r^   rS   rS   rU   test_check_rebuild_manifest(  s    
z5ArchiveAuthenticationTest.test_check_rebuild_manifestc             C   s   | j dd| j | jd | j dd| j}t| jdd}| | j|d W d Q R X | j dd	| j}| j d
d| j | j dd	| j}| j dd| j}d S )Nr   z--encryption=repokeyr  rl   z--format="{name} {id}{NL}"T)r   r  z--format="{name} tam:{tam}{NL}"r   z--repair)r   r   r   r:   r   r  )r   Zarchive_id_pre_checkr   r^   Zarchive_id_post_checkrS   rS   rU   test_check_rebuild_refcounts5  s    
z6ArchiveAuthenticationTest.test_check_rebuild_refcountsN)r  r  r  r  r  r  r  rS   rS   rS   rU   r    s   r  c               @   s\   e Zd ZdZdd Zdd Zdd Zejdd	d
 Z	ejddd Z
dd Zdd ZdS )RemoteArchiverTestCasez__testsuite__:c             C   s   t t| jS )N)r8   r%   r   )r   rS   rS   rU   r   J  s    z&RemoteArchiverTestCase.open_repositoryc          6   C   s<  t jtdd| jg | jdd| j W d Q R X t jtdd| jg0 tjt | jdd| jd  W d Q R X W d Q R X t jtdddg0 tjt | jdd| jd  W d Q R X W d Q R X t	j
j| j}t jtdd|g | jdd| jd  W d Q R X t jtdddd|g | jdd| jd	  W d Q R X d S )
Nextra_test_argsz--restrict-to-pathr   z--encryption=repokeyZ_0z/fooZ_1Z_2Z_3)r   r  r8   r   r   r   r%  r&  r9   rb   r   r   )r   r
  rS   rS   rU   !test_remote_repo_restrict_to_pathM  s    ((z8RemoteArchiverTestCase.test_remote_repo_restrict_to_pathc             C   s   t jtdd| jg | jdd| j W d Q R X tjj| jd}t jtdd|g, t	j
t | jdd| j W d Q R X W d Q R X d S )Nr  z--restrict-to-repositoryr   z--encryption=repokeyz..)r   r  r8   r   r   r   rb   r   r   r%  r&  r9   )r   r'  rS   rS   rU   'test_remote_repo_restrict_to_repositoryb  s    z>RemoteArchiverTestCase.test_remote_repo_restrict_to_repositoryzonly works locallyc             C   s   d S )NrS   )r   rS   rS   rU   rn  k  s    z4RemoteArchiverTestCase.test_debug_put_get_delete_objc             C   s   d S )NrS   )r   rS   rS   rU   r  o  s    z"RemoteArchiverTestCase.test_configc          "   C   s(  | j dd| j | jddd | jddd | jdd	d | jd
dd | jddd | j d| jd d d}td | j dd| jd dd}| jd | j dd| jd dd}W d Q R X | jd | j dd| jd dd}W d Q R X | jd | j dd| jd dd}W d Q R X W d Q R X d S )Nr   z--encryption=repokeyzdir/files   test file contents 1)r   z	dir/file2s   test file contents 2zskipped-file1s   test file contents 3zskipped-file2s   test file contents 4zskipped-file3s   test file contents 5r   z::testrN   z)cached responses left in RemoteRepositoryr^   r   z--debugz--strip-componentsrw  r3  rx  ry  zinput/dir/filerz  )r   r   r   r?   r{  )r   ZmarkerresrS   rS   rU   !test_strip_components_doesnt_leaks  s&    


z8RemoteArchiverTestCase.test_strip_components_doesnt_leakc             C   s<   | j d d }| jd|ddd}| jd| | jd| d	S )
z.https://github.com/borgbackup/borg/issues/6014z-this-repository-does-not-existz::testr  r   T)rj   rL   zthis-repository-does-not-existz$this-repository-does-not-exist::testN)r   r   r  r  )r   r   r^   rS   rS   rU   4test_do_not_mention_archive_if_you_can_not_find_repo  s    zKRemoteArchiverTestCase.test_do_not_mention_archive_if_you_can_not_find_repoN)r  r  r  r   r   r  r   r  r*  rn  r  r  r  rS   rS   rS   rU   r  G  s   	r  c                   sF   e Zd Z fddZdddZdd Zdd	 Zd
d Zdd Z  Z	S )ArchiverCorruptionTestCasec                sF   t  j  | j  | jdd| j tj| jd| jdd d | _d S )Nr   z--encryption=repokeyr  z--jsonr   r   )r  r   r  r   r   r  r  r   )r   )r  rS   rU   r     s    
z ArchiverCorruptionTestCase.setUpr;   c             C   s\   t |dH}|j| tj tdd |j|D }|j| tj |j| W d Q R X d S )Nzr+bc             s   s   | ]}d | V  qdS )   NrS   )r   crS   rS   rU   r     s    z5ArchiverCorruptionTestCase.corrupt.<locals>.<genexpr>)r   rU  rh   SEEK_ENDbytesr0  r   )r   r3  Zamountr   Z	corruptedrS   rS   rU   corrupt  s
    z"ArchiverCorruptionTestCase.corruptc          
   C   sX   | j tjj| jd | jr0| jd| jdd}n$tj	t
 | jd| j W d Q R X d S )Nr  r  r   )rj   )r	  rb   r   r   r   r   r   r   r%  r&  r$   )r   r   rS   rS   rU   test_cache_chunks  s    z,ArchiverCorruptionTestCase.test_cache_chunksc             C   sB   | j d| jd d | jtjj| jd | j d| jd d}d S )Nr   z::testrN   rW  z::test1)r   r   r	  rb   r   r   r   )r   r   rS   rS   rU   test_cache_files  s    z+ArchiverCorruptionTestCase.test_cache_filesc             C   s   | j d| jd d | j d| jdj }| j d| jd d | j dd| j | j d	| jd
 tjj| jd}| jtjj||d  tjj| jd}td d}|j	| |j
ddttd t|d}|j| W d Q R X | j dd| jd ddd}d S )Nr   z::test1rN   rl   z--format={id}{LF}z::test2rk  z--cache-onlyr  z--jsonzchunks.archive.dz.compactrK  )rJ  r   r       rL  z-vz::test3r;   )rj   )r   r   r  rb   r   r   r   r	  r   r0  rM  r-   r  r   r   )r   Z	target_idZchunks_archiveconfig_pathrK  r   r   rS   rS   rU   test_chunks_archive  s    

z.ArchiverCorruptionTestCase.test_chunks_archivec          
   C   sl   t jj| jd}td d}|j| |jddttd t	|d}|j
| W d Q R X | jd| j}d S )NrK  )rJ  r   r   r  rL  r  )rb   r   r   r   r   r0  rM  r-   r  r   r   r   r   )r   r  rK  r   r   rS   rS   rU   test_old_version_interfered  s    

z6ArchiverCorruptionTestCase.test_old_version_interfered)r;   )
r  r  r  r   r	  r
  r  r  r  r  rS   rS   )r  rU   r    s   

r  c               @   s   e Zd Zdd Zdd ZdS )DiffArchiverTestCasec             C   s  | j   | jdd| j | jddd | jddd | jdd	d | jd
dd tjd tjdtjdB  tjd t	 rtjd tj
dd tj
dd tj
dd tj
dd tj
dd t rtjdd tjdd tjdd tjdd | jd| jd d  | jd!d"d | jd#d$d tjd | jd
d%d& d' tjd tjd tjd | jd(d)d tjdtjdB  tjd* tjd t	 rtjd tj
d*d tjd tj
d*d tj
d*d+ tjd | jd,d-d tjd t rtjd tjd.d/ tdd0}|jd1 W d Q R X | jd| jd2 d  | jdd3d4| jd5 d  d6d7 }d8d9 }|| jd:| jd d;d< || jd:| jd d=d>d?d@ || jd:| jd d;dAd< d S )BNr   z--encryption=repokeyZfile_unchanged   )r   Zfile_removed   Zfile_removed2i   Zfile_replacedi   zinput/dir_replaced_with_filei  zinput/dir_removedzinput/dir_replaced_with_linkzinput/link_changedzinput/file_unchangedzinput/link_removedzinput/file_removed2zinput/link_target_removedzinput/emptyz"input/link_target_contents_changedzinput/link_replaced_by_filezinput/file_replacedzinput/hardlink_target_replacedzinput/hardlink_contents_changedzinput/file_removedzinput/hardlink_removedzinput/hardlink_target_removedr   z::test0rN   Z
file_addedi   Zfile_empty_addedr   r  i   )r   Zdir_replaced_with_filei    zinput/dir_addedzinput/link_addedZlink_replaced_by_filei @  zinput/file_addedzinput/hardlink_addedZabs   appended_dataz::test1az--chunker-paramsz16,18,17,4095z::test1bc             S   s\   |rdndj d}dtjkrt r"|r*dndj d}t r:t r@t rFt rLt rRt rXd S )NBz{:<19}modifiedr  z0 B)formatrb   r   rB   rC   )r^   can_compare_idsZchangerS   rS   rU   
do_asserts  s    
	zADiffArchiverTestCase.test_basic_functionality.<locals>.do_assertsc             S   s   dd }dd | j dD }|r,dddd	nd
di}dtjkr>t rD|rTdddd	nd
di}t rbt rht rnt rtt rzt rd S )Nc                s     fdd|D }dd |D S )Nc                s    g | ]}|d   kr|d qS )r   ZchangesrS   )r   r   )r   rS   rU   r  ]  s    zoDiffArchiverTestCase.test_basic_functionality.<locals>.do_json_asserts.<locals>.get_changes.<locals>.<listcomp>c             S   s   g | ]}|D ]}|qqS rS   rS   )r   ZchgsetZchgrS   rS   rU   r  `  s    rS   )r   r   ZchgsetsrS   )r   rU   get_changes\  s    z[DiffArchiverTestCase.test_basic_functionality.<locals>.do_json_asserts.<locals>.get_changesc             S   s   g | ]}|rt j|qS rS   )r  r  )r   r   rS   rS   rU   r  c  s    zZDiffArchiverTestCase.test_basic_functionality.<locals>.do_json_asserts.<locals>.<listcomp>r  r  i   i   )r1  ZaddedZremovedr1  r     r   )r  rb   r   rB   rC   )r^   r  r  Zjoutputr  rS   rS   rU   do_json_asserts[  s    
	zFDiffArchiverTestCase.test_basic_functionality.<locals>.do_json_assertsr  Ztest1aTZtest1br;   )rj   Fz--json-lines)r  r   r   r   rb   r   r   r   S_IFDIRrB   r   rC   r   r  rmdirS_IFREGr   r   )r   r   r  r  rS   rS   rU   r    sl    













BMz-DiffArchiverTestCase.test_basic_functionalityc             C   s   | j dd| j | jddd | jddd | jdd	d | jd
dd | j d| jd d tjd tjd tjd tjd | jddd | jd
dd | jddd | jddd | j d| jd d | j dd| jd d}ddddd
dg}d S )Nr   z--encryption=repokeyZa_file_removed   )r   Zf_file_removed   Zc_file_changedr  Ze_file_changedr  r   z::test0rN   zinput/a_file_removedzinput/f_file_removedzinput/c_file_changedzinput/e_file_changedi   i   Zb_file_addedr  Zd_file_addedr  z::test1r  z--sortr  )r   r   r   rb   r  )r   r^   r  rS   rS   rU   test_sort_option  s.    



z%DiffArchiverTestCase.test_sort_optionN)r  r  r  r  r   rS   rS   rS   rU   r    s    Rr  c              C   s   t  } | jddddgd}| jddddgd}| jddddgd}| jddddgd	}| jddddgd
}| jddddgd}| jddgd}d S )Nrp   Zservez--restrict-to-path=/p1z--restrict-to-path=/p2zborg serve --info --umask=0027z--restrict-to-repository=/r1z--restrict-to-repository=/r2zborg serve --restrict-to-path=/z%borg serve --restrict-to-repository=/z-borg serve --restrict-to-repository=/r1/belowz borg init --encryption=repokey /z-BORG_HOSTNAME_IS_UNIQUE=yes borg serve --info)r   Zget_args)rK   rT   rS   rS   rU   test_get_args  s     
r!  c              C   s   dd } d S )Nc             S   sH   dd | D }dd |D }t jt|t|}t jt|t|}|S )Nc             S   s   g | ]}|qS rS   rS   )r   r   rS   rS   rU   r    s    z<test_compare_chunk_contents.<locals>.ccc.<locals>.<listcomp>c             S   s   g | ]}|qS rS   rS   )r   r   rS   rS   rU   r    s    )r   Zcompare_chunk_contentsiter)r  r  Zchunks_aZchunks_bZcompare1Zcompare2rS   rS   rU   ccc  s
    z(test_compare_chunk_contents.<locals>.cccrS   )r#  rS   rS   rU   test_compare_chunk_contents  s    r$  c               @   s0   e Zd Zedd Zdd Zdd Zdd Zd	S )
TestBuildFilterc             C   s   d S )NrS   )r  ZmatchedrS   rS   rU   peek_and_store_hardlink_masters  s    z/TestBuildFilter.peek_and_store_hardlink_mastersc             C   s.   t  }|jtdgtj tj|| jd}d S )NZincludedr   )r3   r^  r4   r2   ZIncluder   build_filterr&  )r   matcherr  rS   rS   rU   
test_basic  s    zTestBuildFilter.test_basicc             C   s   t dd}tj|| jd}d S )NT)fallbackr   )r3   r   r'  r&  )r   r(  r  rS   rS   rU   
test_empty#  s    
zTestBuildFilter.test_emptyc             C   s    t dd}tj|| jdd}d S )NT)r*  r;   )Zstrip_components)r3   r   r'  r&  )r   r(  r  rS   rS   rU   r|  (  s    
z%TestBuildFilter.test_strip_componentsN)r  r  r  staticmethodr&  r)  r+  r|  rS   rS   rS   rU   r%    s   r%  c               @   s   e Zd Zedd Zejdd Zejdd Zejdd Z	ejd	d
 Z
ejdd Zdd Zejjddejjdd dd ZdS )!TestCommonOptionsc             C   sv   | ddddd | dddd	d
dd | dddd	ddd | dddddg d | dddddd | ddt dddd d S )Nz-hz--helpr{   zshow this help message and exit)actionr{   z
--critical	log_levelr  store_constZcriticalwarning)destr{   r.  constr  z--errorerrorz--appendr  ZTOPIC)r2  r{   r.  metavarr  z-pz
--progressprogress
store_true)r2  r.  r{   z--lock-wait	lock_waitNr;   z(default: %(default)d).)r2  r1  r5  r  r{   )r<  )Zadd_common_optionrS   rS   rU   define_common_options2  s    z'TestCommonOptions.define_common_optionsc             C   s&   t jdddd}tj| jdd|_|S )	Nr
  ztest parserF)progdescriptionadd_help_level0_level1)Zsuffix_precedence)r>  r?  )argparseArgumentParserr   ZCommonOptionsr:  common_options)r   parserrS   rS   rU   basic_parser?  s    
zTestCommonOptions.basic_parserc             C   s   |j dddS )Nzrequired argumentsz	<command>)titler5  )Zadd_subparsers)r   rD  rS   rS   rU   
subparsersF  s    zTestCommonOptions.subparsersc             C   s   |j j|ddd |S )Nr>  T)Zprovide_defaults)rB  add_common_group)r   rD  rS   rS   rU   rC  J  s    zTestCommonOptions.parserc             C   s    t jddd}|jj|d |S )NFr
  )r=  r;  r?  )r@  rA  rB  rG  )r   rC  common_parserrS   rS   rU   rH  O  s    zTestCommonOptions.common_parserc          	      sH   |j d|gddddtjd}|jdd |jd	d
dd  fdd}|S )N
subcommandFr  r  Zbaz)parentsr=  r<  epilogr{   Zformatter_classi  )funcz--append-onlyr  r7  )r2  r.  c                 s&   t |   j| } jj| t|S )N)r   rk   rB  Zresolvevars)r   rT   )rC  rS   rU   parse_vars_from_line]  s    
zDTestCommonOptions.parse_vars_from_line.<locals>.parse_vars_from_line)Z
add_parserr@  ZRawDescriptionHelpFormatterZset_defaultsadd_argument)r   rC  rF  rH  Z	subparserrN  rS   )rC  rU   rN  U  s    
z&TestCommonOptions.parse_vars_from_linec          
   C   s$   t jt |dd W d Q R X d S )Nz--append-onlyrI  )r%  r&  r`   )r   rN  rS   rS   rU   test_simplee  s    zTestCommonOptions.test_simplepositionru  rw  bothzflag,args_key,args_value-pr6  T--lock-wait=3r8     c             C   sP   g }|d
kr|j | |j d |dkr2|j | g dddddd	}|||< d S )Nru  rR  rI  rw  r;   r1  Fi  )r  r8  r/  r6  r  rL  )ru  rR  )rw  rR  )r  )r   rN  rQ  flagZargs_keyZ
args_valuer   r  rS   rS   rU   test_flag_position_independence  s    


z1TestCommonOptions.test_flag_position_independenceN)ru  rw  rR  rS  r6  TrT  r8  rU  )rX  rY  )r  r  r  r,  r:  r%  fixturerD  rF  rC  rH  rN  rP  r  parametrizerW  rS   rS   rS   rU   r-  1  s    r-  c            	   C   s$   t jtj td W d Q R X d S )NZ5M)r%  r&  r@  ZArgumentTypeErrorr   rS   rS   rS   rU   test_parse_storage_quota  s    r\  c                 sF   t ddj } t ddj }i d fdd	  d| t d|i S )	z0
    Return dict mapping command to parser.
    rp   )r;  ZborgfsNc                s   i }xJ|j D ]@}|jd k	rdt|jkrx"|jj D ]\}}||| | < q4W qW |d k	rb|j| | rp| rpd S x0t|j D ] \}} |d || ||< q~W d S )NZSubParsersActionr+  )Z_actionschoicesrZ   r  r  updater  )r   rC  r   Zextra_choicesr]  r.  r   command)discover_levelparsersrS   rU   r`    s    

z'get_all_parsers.<locals>.discover_levelrQ   )N)r   Zbuild_parser)rC  Zborgfs_parserrS   )r`  ra  rU   get_all_parsers  s    rb  zcommand, parserc             C   s   t |jtrd S )N)rY   rK  r0   )r_  rC  rS   rS   rU   test_help_formatting  s    rc  ztopic, helptextc             C   s   d S )NrS   )ZtopichelptextrS   rS   rU   test_help_formatting_helptexts  s    re  )r@  r   rh   r  r  rb   r8  r   ru   r)  r   r]   rW   r   r   rer  Zbinasciir   r   Zconfigparserr   r   r   r   Zhashlibr   r	   r
   Zunittest.mockr   r%  ZllfuseImportErrorrp   Zborg.helpersrQ   r   r   r   r   r   r   r   r   rK   r   r   r   r   r   r   Z	constantsZcrypto.low_levelr   r   Z
crypto.keyr   r   r   r   r    r!   Zcrypto.keymanagerr"   r#   Zcrypto.file_integrityr$   r%   r&   r'   r(   r)   r*   r+   r,   r-   r.   r/   Znanorstr0   r1   r   r2   r3   r4   r  r5   Zlockingr6   Zloggerr7   Zremoter8   r9   r   r:   r<   r=   r>   r?   r@   rA   rB   rC   rD   rE   rF   rG   ZupgraderrH   rI   r   r  r   r   __file__r   rs   rz   Z	BORG_EXESFileNotFoundErrorrZ  r   r   r   r  r  r   r   r   r  r  r  r  r  r  r  r  r  r!  r$  r%  r-  r\  rb  r[  rl   r  rc  rd  re  rS   rS   rS   rU   <module>   s    1	 &?                       0+ tL@JJ s(!k 