
­­­­­­­­­­­­­­­­­­
<!DOCTYPE html>
<html>
3
yb                 @   s  d dl Z d dlZd dlZd dlZd dlZd dlZd dlmZ d dlZddl	m
Z
 ddlmZ ddlmZ ddlmZ ddlmZmZ dd	lmZmZmZmZmZ dd
lmZmZmZmZmZmZmZ ddl m!Z! ddl	m"Z" e# Z$G dd de!Z%G dd de%Z&G dd de%Z'G dd de%Z(G dd de%Z)G dd de%Z*G dd de%Z+G dd de%Z,G dd de%Z-G d d! d!e%Z.G d"d# d#e%Z/G d$d% d%e&Z0G d&d' d'e%Z1G d(d) d)e.Z2G d*d+ d+e!Z3dS ),    N)patch   )NSIndex)Location)IntegrityError)msgpack)Lock
LockFailed)RemoteRepositoryInvalidRPCMethodPathNotAllowedConnectionClosedWithHinthandle_remote_line)
RepositoryLoggedIOMAGICMAX_DATA_SIZE
TAG_DELETETAG_PUT
TAG_COMMIT   )BaseTestCase)Hc               @   sP   e Zd ZdZdZdefddZdd Zdd	 Zefd
dZ	dd Z
dddZdS )RepositoryTestCaseBase    TFc             C   s(   |t kr| j}ttjj| jd||dS )N
repository)	exclusivecreate)UNSPECIFIEDr   r   ospathjointmppath)selfr   r    r$   "/usr/lib64/python3.6/repository.pyopen   s    zRepositoryTestCaseBase.openc             C   s&   t j | _| jdd| _| jj  d S )NT)r   )tempfileZmkdtempr"   r&   r   	__enter__)r#   r$   r$   r%   setUp"   s    
zRepositoryTestCaseBase.setUpc             C   s   | j j  tj| j d S )N)r   closeshutilZrmtreer"   )r#   r$   r$   r%   tearDown'   s    
zRepositoryTestCaseBase.tearDownc             C   s"   | j r| j j  | j|d| _ d S )N)r   )r   r*   r&   )r#   r   r$   r$   r%   reopen+   s    
zRepositoryTestCaseBase.reopenc             C   sx   | j jtdd | j jtdd | j jtdd | j j  | j jtdd | j jtdd | j jtd d S )	Nr   s   foor   s   bar   s   bar2r   s   boo)r   putr   commitdelete)r#   r$   r$   r%   add_keys0   s    
zRepositoryTestCaseBase.add_keysNc       
   
   C   s   |d k	r|d nd}dd t dD }d|d < tdtdtd	i}xX| jjj D ]H\}}x>| jjj|D ],\}}}}	td
||| || |||	f  qdW qLW t  d S )Nz:  c             S   s   i | ]}|t |qS r$   )r   ).0ir$   r$   r%   
<dictcomp>;   s    z4RepositoryTestCaseBase.repo_dump.<locals>.<dictcomp>
   r   r/   delZcommz%s%s H(%d) -> %s[%d..+%d])	ranger   r   r   r   iosegment_iteratoriter_objectsprint)
r#   ZlabelZH_transZ	tag_transsegmentfntagkeyoffsetsizer$   r$   r%   	repo_dump9   s    (z RepositoryTestCaseBase.repo_dump)N)__name__
__module____qualname__Zkey_sizer   r   r&   r)   r,   r-   r2   rE   r$   r$   r$   r%   r      s   	r   c               @   sT   e Zd Zdd Zdd Zdd Zdd Zd	d
 Zdd Zdd Z	dd Z
dd ZdS )RepositoryTestCasec                s   x"t dD ]}jjt|d q
W td jjj d jj  jtj	 fdd jj
  jj  j Pjtj	 fdd x0t dD ]$}|dkrqjjt|d qW W d Q R X d S )Nd   s   SOMEDATA2   c                  s   j j S )N)r   getr$   )key50r#   r$   r%   <lambda>L   s    z*RepositoryTestCase.test1.<locals>.<lambda>c                  s
   j  S )N)rL   r$   )rM   repository2r$   r%   rN   P   s    )r:   r   r/   r   assert_equalrL   r1   assert_raisesr   ObjectNotFoundr0   r*   r&   )r#   xr$   )rM   rO   r#   r%   test1F   s    


zRepositoryTestCase.test1c             C   sv   | j jtdd | j jtdd | j j  | j jtd | j jtdd | j j  | j| j jtdd dS )z.Test multiple sequential transactions
        r   s   foor   s   barN)r   r/   r   r0   r1   rP   rL   )r#   r$   r$   r%   test2V   s    

zRepositoryTestCase.test2c                s    j jtdd  j j jtdd  j jtdd  j j jtdd  j jtdd  j j jtdd  j jtd  jtj fdd dS )zTest cache consistency
        r   s   foos   foo2s   barc                  s    j jtdS )Nr   )r   rL   r   r$   )r#   r$   r%   rN   k   s    z5RepositoryTestCase.test_consistency.<locals>.<lambda>N)	r   r/   r   rP   rL   r1   rQ   r   rR   )r#   r$   )r#   r%   test_consistencya   s    z#RepositoryTestCase.test_consistencyc             C   s   | j jtdd | j| j jtdd | j j  | j jtdd | j| j jtdd | j j  | j| j jtdd dS )z Test cache consistency2
        r   s   foos   foo2N)r   r/   r   rP   rL   r0   rollback)r#   r$   r$   r%   test_consistency2m   s    

z$RepositoryTestCase.test_consistency2c             C   sJ   | j jtdd | j jtdd | j j  | j| j jtdd dS )z Test cache consistency2
        r   s   foos   foo2N)r   r/   r   r0   rP   rL   )r#   r$   r$   r%   "test_overwrite_in_same_transactionx   s    
z5RepositoryTestCase.test_overwrite_in_same_transactionc             C   s   | j jtdd | j j  | j j  | j | _ | j " | j jtdd | j j  W d Q R X | j | _ | j   | j jtd | j j  W d Q R X d S )Nr   s   foos   bar)r   r/   r   r0   r*   r&   r1   )r#   r$   r$   r%   test_single_kind_transactions   s    



z0RepositoryTestCase.test_single_kind_transactionsc             C   s   x"t dD ]}| jjt|d q
W | jj  | jj }| jt|d | jjdd}| jt|d | j||d d  | jj|d d}| jt|d | j||dd   | jt| jjddd d S )NrJ   s   SOMEDATArK   )limitr   )markerr9   )r:   r   r/   r   r0   listrP   len)r#   rS   all
first_halfsecond_halfr$   r$   r%   	test_list   s    

zRepositoryTestCase.test_listc             C   s   x"t dD ]}| jjt|d q
W | jj  | jj }t|dksHt| jjdd}t|dksft||d d kszt| jj|d d}t|dkst||dd  kstt| jjdddkstx$t dD ]}|| t|kstqW d S )NrJ   s   SOMEDATArK   )r[   r   )r\   r9   )r:   r   r/   r   r0   scanr^   AssertionError)r#   rS   r_   r`   ra   r$   r$   r%   	test_scan   s    

zRepositoryTestCase.test_scanc                sL   dt   jjtd  jjjtd  jt fdd d S )N   xr   c                  s   j jtd d S )Nr   rf   )r   r/   r   r$   )max_datar#   r$   r%   rN      s    z7RepositoryTestCase.test_max_data_size.<locals>.<lambda>)r   r   r/   r   rP   rL   rQ   r   )r#   r$   )rg   r#   r%   test_max_data_size   s
    z%RepositoryTestCase.test_max_data_sizeN)rF   rG   rH   rT   rU   rV   rX   rY   rZ   rb   re   rh   r$   r$   r$   r%   rI   D   s   rI   c               @   s4   e Zd Zdd Zdd Zdd Zdd Zd	d
 ZdS )LocalRepositoryTestCasec             C   sL   | j jd dkst| j jd dks(t| j jd | j jd dksHtd S )Nr   )   	   r   rK   rK   )r   compactrd   _rebuild_sparse)r#   r$   r$   r%   _assert_sparse   s    z&LocalRepositoryTestCase._assert_sparsec             C   sL   | j jtdd | j jtdd | j j  | j jtdd | j  d S )Nr   s   foor   s	   123456789s   bar)r   r/   r   r0   rn   )r#   r$   r$   r%   test_sparse1   s
    
z$LocalRepositoryTestCase.test_sparse1c             C   sJ   | j jtdd | j jtdd | j j  | j jtd | j  d S )Nr   s   foor   s	   123456789)r   r/   r   r0   r1   rn   )r#   r$   r$   r%   test_sparse2   s
    
z$LocalRepositoryTestCase.test_sparse2c             C   s   | j jtdd | j jtd | j jjj  | j jd dksDt| j j	d | j jd d
t
t kslt| j j  ddd | j jj D kstd S )Nr   s   1245rj      c             S   s   g | ]\}}|qS r$   r$   )r4   r?   _r$   r$   r%   
<listcomp>   s    z>LocalRepositoryTestCase.test_sparse_delete.<locals>.<listcomp>R   V   rt   ru   )r   r/   r   r1   r;   Z	_write_fdsyncrl   rd   rm   r^   r   r0   r<   )r#   r$   r$   r%   test_sparse_delete   s    
z*LocalRepositoryTestCase.test_sparse_deletec             C   s   | j jtdd | j j  | j jj }t| j jj|d d}|jt	d  W d Q R X | j j
  | j | _ | j " | j jtdd | j j  W d Q R X d S )Nr   s   foor   wbs   crapcrapcraps   bar)r   r/   r   r0   r;   get_latest_segmentr&   Zsegment_filenamewriter   r*   )r#   last_segmentfr$   r$   r%   test_uncommitted_garbage   s    


z0LocalRepositoryTestCase.test_uncommitted_garbageN)rF   rG   rH   rn   ro   rp   rw   r}   r$   r$   r$   r%   ri      s
   ri   c               @   s\   e Zd Zdd Zdd Zdd Zdd Zd	d
 Zdd Zdd Z	dd Z
dd Zdd ZdS )RepositoryCommitTestCasec          
   C   s   | j   x8tj| jjD ]&}|jdrtjtjj| jj| qW | j  | j* | j	t
| jd | j	| jj d W d Q R X d S )Nzindex.r.   T)r2   r   listdirr   r    
startswithunlinkr!   r-   rP   r^   check)r#   namer$   r$   r%   test_replay_of_missing_index   s    
z5RepositoryCommitTestCase.test_replay_of_missing_indexc             C   sv   | j   d | j_y| jj  W n tk
r2   Y nX | j  | j* | jt| jd | j| jj d W d Q R X d S )Nr.   T)	r2   r   compact_segmentsr0   	TypeErrorr-   rP   r^   r   )r#   r$   r$   r%   "test_crash_before_compact_segments   s    z;RepositoryCommitTestCase.test_crash_before_compact_segmentsc             C   sv   | j   d | j_y| jj  W n tk
r2   Y nX | j  | j* | jt| jd | j| jj d W d Q R X d S )Nr.   T)	r2   r   Zwrite_indexr0   r   r-   rP   r^   r   )r#   r$   r$   r%   test_crash_before_write_index  s    z6RepositoryCommitTestCase.test_crash_before_write_indexc                s    j   x8tj jjD ]&}|jdrtjtjj jj| qW tj	t
dtd@} jd d  j"  jt fdd |j  W d Q R X W d Q R X d S )Nzindex.upgrade)side_effect)r   c                  s
   t  jS )N)r^   r   r$   )r#   r$   r%   rN     s    zGRepositoryCommitTestCase.test_replay_lock_upgrade_old.<locals>.<lambda>)r2   r   r   r   r    r   r   r!   r   objectr   r	   r-   rQ   assert_called_once_with)r#   r   r   r$   )r#   r%   test_replay_lock_upgrade_old  s    
z5RepositoryCommitTestCase.test_replay_lock_upgrade_oldc                s    j   x8tj jjD ]&}|jdrtjtjj jj| qW tj	t
dtd@} jdd  j"  jt fdd |j  W d Q R X W d Q R X d S )Nzindex.r   )r   F)r   c                  s
   t  jS )N)r^   r   r$   )r#   r$   r%   rN   +  s    zCRepositoryCommitTestCase.test_replay_lock_upgrade.<locals>.<lambda>)r2   r   r   r   r    r   r   r!   r   r   r   r	   r-   rQ   r   )r#   r   r   r$   )r#   r%   test_replay_lock_upgrade!  s    
z1RepositoryCommitTestCase.test_replay_lock_upgradec             C   s   | j   d | jj_y| jj  W n tk
r4   Y nX | j  | j< | jt| jd | j| jj	 d | jt| jd W d Q R X d S )Nr.   T)
r2   r   r;   delete_segmentr0   r   r-   rP   r^   r   )r#   r$   r$   r%   -test_crash_before_deleting_compacted_segments.  s    
zFRepositoryCommitTestCase.test_crash_before_deleting_compacted_segmentsc          	   C   sN   | j jtdtj | j  | j " | j j}|j|j  s@t	W d Q R X d S )Nr   )
r   r/   r   r   ZCOMMITr-   r;   Zis_committed_segmentry   rd   )r#   r;   r$   r$   r%   test_ignores_commit_tag_in_data;  s
    z8RepositoryCommitTestCase.test_ignores_commit_tag_in_datac       	      C   s  | j jtdd | j jtdd | j j  | jd | j jtd | j j  | jd | j jj d }d}x>| j jj|D ],\}}}}|t	kr|tdkst
|d7 }qW |dkst
|| j jkst
| j jtdd	 | j j  | jd
 || j jkst
| j jj| st
x\| j jj D ]L\}}x@| j jj|D ].\}}}}|t	ksZt
|tdks@t
q@W q(W td| j jkst
d S )Nr      1r      2zp1 p2 cczd1 ccr   r.      3zp3 cc)r   r/   r   r0   rE   r1   r;   ry   r=   r   rd   rl   segment_existsr<   shadow_index)	r#   r{   Znum_deletesrA   rB   rC   rD   r?   rr   r$   r$   r%   test_moved_deletes_are_trackedB  s2    





z7RepositoryCommitTestCase.test_moved_deletes_are_trackedc             C   s   | j jj}| j jtdd | }| j j  | j jtd | j jtdd | }| j j|= | j j|= | j j  | j jtd | j j  | j jj|st	| j jj| st	td| j kst	t
jt
jj| j jd|   td| j kst	d S )Nr   r   r   zindex.%d)r   r;   ry   r/   r   r0   r1   rl   r   rd   r   r   r    r!   )r#   ry   Zput_segmentr   r$   r$   r%   #test_shadowed_entries_are_preserved_  s"    





z<RepositoryCommitTestCase.test_shadowed_entries_are_preservedc             C   s   | j jtdd | j jtd | j jtd dgks<t| j j  | jd | j jtd g ksht| j jtdd | j jtd | jd | j jtd dgkst| j j  | jd | j jtdd | j jtd g kstd S )	Nr   r   r   zp1 d1 cczp1 d1rq   rr   )	r   r/   r   r1   r   rd   r0   rE   rW   )r#   r$   r$   r%   test_shadow_index_rollback  s    




z3RepositoryCommitTestCase.test_shadow_index_rollbackN)rF   rG   rH   r   r   r   r   r   r   r   r   r   r   r$   r$   r$   r%   r~      s   
"r~   c               @   s&   e Zd Zd	ddZdd Zdd ZdS )
RepositoryAppendOnlyTestCaseFc             C   s   t tjj| jdd|ddS )Nr   T)r   r   append_only)r   r   r    r!   r"   )r#   r   r$   r$   r%   r&     s    z!RepositoryAppendOnlyTestCase.openc          	   C   s0   | j t | jj  W d Q R X | jjs,td S )N)rQ   
ValueErrorr   Zdestroyr   rd   )r#   r$   r$   r%   test_destroy_append_only  s    z5RepositoryAppendOnlyTestCase.test_destroy_append_onlyc                s    fdd} j jtdd  j j  d j _| dks>t j jtdd  j j  | dkshtd j _| dks~t j jtdd  j j  | d	kstd S )
Nc                  s   t t jjj S )N)r^   r]   r   r;   r<   r$   )r#   r$   r%   segments_in_repository  s    zMRepositoryAppendOnlyTestCase.test_append_only.<locals>.segments_in_repositoryr   s   fooFr   rq   T   )r   r/   r   r0   r   rd   )r#   r   r$   )r#   r%   test_append_only  s    


z-RepositoryAppendOnlyTestCase.test_append_onlyN)F)rF   rG   rH   r&   r   r   r$   r$   r$   r%   r     s   
r   c               @   s   e Zd Zdd Zdd ZdS )RepositoryFreeSpaceTestCasec             C   s   | j   | jjjddd | jjd | j  | j: | jjtdd tj	t
j | jj  W d Q R X W d Q R X tjj| jjstd S )Nr   additional_free_spaceZ1000Ts   shortcut to save_configr   s   foobar)r2   r   configsetZsave_keyr-   r/   r   pytestraisesr   InsufficientFreeSpaceErrorr0   r   r    existsrd   )r#   r$   r$   r%   test_additional_free_space  s    z6RepositoryFreeSpaceTestCase.test_additional_free_spacec          	   C   sB   d| j _tjtj | j  W d Q R X tjj	| j j s>t
d S )Ng@xD)r   r   r   r   r   r   r2   r   r    r   rd   )r#   r$   r$   r%   test_create_free_space  s    z2RepositoryFreeSpaceTestCase.test_create_free_spaceN)rF   rG   rH   r   r   r$   r$   r$   r%   r     s   r   c               @   s   e Zd Zdd Zdd ZdS )QuotaTestCasec             C   s  | j jdkst| j jtdtd | j jd	ks6t| j jtdtd | j jdks\t| j jtd | j jdks|t| j j  | j jdkst| j  | j V | j jtdd | j jtd | j jdk st| j j  | j jdk stW d Q R X d S )Nr   r   i  rj   r   i.  r.       i  i   rt   iR  i   rt   iR  iW  rt   i  iW  )	r   storage_quota_userd   r/   r   bytesr1   r0   r-   )r#   r$   r$   r%   test_tracking  s     

zQuotaTestCase.test_trackingc             C   s  | j jdkstd| j _| j jtdd | j jdks:t| j j  tjt	j
 | j jtdd W d Q R X | j jdks~ttjt	j
 | j j  W d Q R X | j jdkst| j  | j L d| j _| j jtdd | j jdkst| j j  | j jdkstW d Q R X d S )	Nr   rK   r   r   rj   r   rt   rJ   )r   r   rd   storage_quotar/   r   r0   r   r   r   ZStorageQuotaExceededr-   )r#   r$   r$   r%   test_exceed_quota  s$    

zQuotaTestCase.test_exceed_quotaN)rF   rG   rH   r   r   r$   r$   r$   r%   r     s   r   c               @   s,   e Zd Zdd Zdd Zdd Zdd Zd	S )
NonceReservationc             C   sB   | j dd tjt" | j | jj  W d Q R X W d Q R X d S )NF)r   )r-   r   r   rd   r   get_free_nonce)r#   r$   r$   r%   test_get_free_nonce_asserts  s    z,NonceReservation.test_get_free_nonce_assertsc             C   s   | j  | j j d kstttjj| j jdd}|jd W d Q R X | j j dksZtttjj| j jdd}|jd W d Q R X | j j dkstW d Q R X d S )NnoncewZ0000000000000000r   Z5000000000000000l            )r   r   rd   r&   r   r    r!   rz   )r#   fdr$   r$   r%   test_get_free_nonce  s    z$NonceReservation.test_get_free_noncec             C   sF   | j dd tjt& | j | jjdd W d Q R X W d Q R X d S )NF)r   i      )r-   r   r   rd   r   commit_nonce_reservation)r#   r$   r$   r%   %test_commit_nonce_reservation_asserts  s    z6NonceReservation.test_commit_nonce_reservation_assertsc          %   C   s   | j  tjt | j jdd W d Q R X | j jdd  ttjj| j jdd}|j	 dksdt
W d Q R X tjt | j jdd W d Q R X | j jdd ttjj| j jdd}|j	 dkst
W d Q R X W d Q R X d S )Ni      r   r   Z0000000000000200i   Z0000000000000400)r   r   r   	Exceptionr   r&   r   r    r!   readrd   )r#   r   r$   r$   r%   test_commit_nonce_reservation  s    z.NonceReservation.test_commit_nonce_reservationN)rF   rG   rH   r   r   r   r   r$   r$   r$   r%   r     s   r   c                   s   e Zd Z fddZdd Zdd Zdd Zd	d
 Zdd Zdd Z	dd Z
dd Zdd Zdd Zdd Zdd Zdd Zdd Zdd  Z  ZS )!%RepositoryAuxiliaryCorruptionTestCasec                s4   t  j  | jjtdd | jj  | jj  d S )Nr   s   foo)superr)   r   r/   r   r0   r*   )r#   )	__class__r$   r%   r)     s    

z+RepositoryAuxiliaryCorruptionTestCase.setUpc          
   C   s2   | j " | j jtdd | j j  W d Q R X d S )Nr   s   fox)r   r/   r   r0   )r#   r$   r$   r%   	do_commit#  s    z/RepositoryAuxiliaryCorruptionTestCase.do_commitc          	   C   s:   t tjj| jjdd}|jd W d Q R X | j  d S )Nzhints.1Zabs	   123456789)r&   r   r    r!   r   rz   r   )r#   r   r$   r$   r%   test_corrupted_hints(  s    z:RepositoryAuxiliaryCorruptionTestCase.test_corrupted_hintsc             C   s$   t jt jj| jjd | j  d S )Nzhints.1)r   r   r    r!   r   r   )r#   r$   r$   r%   test_deleted_hints-  s    z8RepositoryAuxiliaryCorruptionTestCase.test_deleted_hintsc             C   s$   t jt jj| jjd | j  d S )Nzindex.1)r   r   r    r!   r   r   )r#   r$   r$   r%   test_deleted_index1  s    z8RepositoryAuxiliaryCorruptionTestCase.test_deleted_indexc          	   C   sH   t jj| jjd}t j| t j| | jt | j  W d Q R X d S )Nzhints.1)	r   r    r!   r   r   mkdirrQ   OSErrorr   )r#   hintsr$   r$   r%   test_unreadable_hints5  s
    

z;RepositoryAuxiliaryCorruptionTestCase.test_unreadable_hintsc          	   C   s:   t tjj| jjdd}|jd W d Q R X | j  d S )Nzindex.1rx   s	   123456789)r&   r   r    r!   r   rz   r   )r#   r   r$   r$   r%   
test_index<  s    z0RepositoryAuxiliaryCorruptionTestCase.test_indexc             C   sV   t tjj| jjdd}|jd W d Q R X | j t| jdksHtW d Q R X d S )Nzindex.1rx   s	   123456789r   )r&   r   r    r!   r   rz   r^   rd   )r#   r   r$   r$   r%   test_index_outside_transactionA  s    zDRepositoryAuxiliaryCorruptionTestCase.test_index_outside_transactionc          
   C   s   t jj| jjd}t|dn}|j }tjtdddA j	dd}|j
td|}||ks^tt|t|ksrt|jd |j| W d Q R X d S )Nzindex.1zr+br   littler   r   )r   r    r!   r   r&   r   int
from_bytesr   to_bytesreplacerd   r^   seekrz   )r#   Z
index_pathr   Z
index_dataZcorrupted_keyZcorrupted_index_datar$   r$   r%   _corrupt_indexG  s    
z4RepositoryAuxiliaryCorruptionTestCase._corrupt_indexc          
   C   sH   | j   | j0 t| jdks"t| jjtddks:tW d Q R X d S )Nr   r   s   foo)r   r   r^   rd   rL   r   )r#   r$   r$   r%   test_index_corruptedU  s    z:RepositoryAuxiliaryCorruptionTestCase.test_index_corruptedc             C   st   | j   tjj| jjd}tj| | j@ t| jdks>ttj	t
j | jjtd W d Q R X W d Q R X d S )Nzintegrity.1r   r   )r   r   r    r!   r   r   r^   rd   r   r   r   rR   rL   r   )r#   integrity_pathr$   r$   r%   &test_index_corrupted_without_integrity_  s    
zLRepositoryAuxiliaryCorruptionTestCase.test_index_corrupted_without_integrityc          	   C   sH   t jj| jjd}t j| t j| | jt | j  W d Q R X d S )Nzindex.1)	r   r    r!   r   r   r   rQ   r   r   )r#   indexr$   r$   r%   test_unreadable_indexk  s
    

z;RepositoryAuxiliaryCorruptionTestCase.test_unreadable_indexc             C   s   t jj| jjd}t|d}tjddi| |j  W d Q R X | j0 t| jdksZt	| jj
tddksrt	W d Q R X d S )Nzintegrity.1zr+bs   versiong@r   r   s   foo)r   r    r!   r   r&   r   packtruncater^   rd   rL   r   )r#   r   r   r$   r$   r%   test_unknown_integrity_versionr  s    zDRepositoryAuxiliaryCorruptionTestCase.test_unknown_integrity_versionc             C   s  | j  d| j _t| j dks"t| j jtddks:t| j jtdd | j jtdd | j j  | j jtdd | j j  W d Q R X tj	j
| j j	d	}t|d
N}tj|}|jd |d d dkstd|d d< tj|| |j  W d Q R X d S )NTr   r   s   foos   barr   s   bazs   bazzzhints.5zr+bs   segments)r   r   r^   rd   rL   r   r/   r0   r   r    r!   r&   r   unpackr   r   r   )r#   Z
hints_pathr   r   r$   r$   r%   _subtly_corrupted_hints_setup  s"    


zCRepositoryAuxiliaryCorruptionTestCase._subtly_corrupted_hints_setupc          
   C   s   | j   | j d| j_| jjtdd | jj  t| jdksFt| jjtddks^t| jjtddksvt| jjtd	d
kstW d Q R X d S )NFr.   s   1234rq   r   s   foor   s   barr   s   bazz)	r   r   r   r/   r   r0   r^   rd   rL   )r#   r$   r$   r%   test_subtly_corrupted_hints  s    
zARepositoryAuxiliaryCorruptionTestCase.test_subtly_corrupted_hintsc             C   s   | j   tjj| jjd}tj| | jR d| j_| jjtdd t	j
t}| jj  W d Q R X dt|jksxtW d Q R X d S )Nzintegrity.5Fr.   s   1234z!Corrupted segment reference count)r   r   r    r!   r   r   r   r/   r   r   r   rd   r0   strvalue)r#   r   exc_infor$   r$   r%   -test_subtly_corrupted_hints_without_integrity  s    
zSRepositoryAuxiliaryCorruptionTestCase.test_subtly_corrupted_hints_without_integrity)rF   rG   rH   r)   r   r   r   r   r   r   r   r   r   r   r   r   r   r   r   __classcell__r$   r$   )r   r%   r     s    
r   c               @   s   e Zd Zdd Zd*ddZdd Zd	d
 Zdd Zdd Zdd Z	dd Z
dd Zdd Zdd Zdd Zdd Zdd Zdd  Zd!d" Zd#d$ Zd%d& Zd'd( Zd)S )+RepositoryCheckTestCasec             C   s    dd t jt jj| jdD S )Nc             S   s   g | ]}|j d r|qS )zindex.)r   )r4   r   r$   r$   r%   rs     s    z8RepositoryCheckTestCase.list_indices.<locals>.<listcomp>r   )r   r   r    r!   r"   )r#   r$   r$   r%   list_indices  s    z$RepositoryCheckTestCase.list_indicesFTc             C   sD   | j | jj|d| | j dd tjtjj| jdD g d d S )N)repairc             S   s   g | ]}d |kr|qS )Ztmpr$   )r4   r   r$   r$   r%   rs     s    z1RepositoryCheckTestCase.check.<locals>.<listcomp>r   zFound tmp files)rP   r   r   r   r   r    r!   r"   )r#   r   statusr$   r$   r%   r     s    zRepositoryCheckTestCase.checkc             G   s"   x|D ]}| j jt| qW d S )N)r   rL   r   )r#   idsid_r$   r$   r%   get_objects  s    
z#RepositoryCheckTestCase.get_objectsc             C   s<   x6|D ].}x|D ]}| j jt|d qW | j j  qW d S )Ns   data)r   r/   r   r0   )r#   segmentsr   r   r$   r$   r%   add_objects  s    

z#RepositoryCheckTestCase.add_objectsc             C   s,   t dd tjtjj| jdddD d S )Nc             s   s   | ]}|j  rt|V  qd S )N)isdigitr   )r4   nr$   r$   r%   	<genexpr>  s    z3RepositoryCheckTestCase.get_head.<locals>.<genexpr>r   data0r   r9   )sortedr   r   r    r!   r"   )r#   r$   r$   r%   get_head  s    z RepositoryCheckTestCase.get_headc             C   s"   t jtjj| jddj| j S )Nr   zindex.{})r   r   r   r    r!   r"   formatr   )r#   r$   r$   r%   
open_index  s    z"RepositoryCheckTestCase.open_indexc          	   C   s\   | j  }|t| \}}ttjj| jdddt|d}|j| |j	d W d Q R X d S )Nr   r   r   zr+bs   BOOM)
r   r   r&   r   r    r!   r"   r   r   rz   )r#   r   idxr?   rC   r   r$   r$   r%   corrupt_object  s
    "
z&RepositoryCheckTestCase.corrupt_objectc             C   s$   t jt jj| jdddt| d S )Nr   r   r   )r   r   r    r!   r"   r   )r#   r?   r$   r$   r%   r     s    z&RepositoryCheckTestCase.delete_segmentc             C   s&   t jt jj| jddj| j  d S )Nr   zindex.{})r   r   r    r!   r"   r   r   )r#   r$   r$   r%   delete_index  s    z$RepositoryCheckTestCase.delete_indexc             C   s6   t jt jj| jddj| j t jj| jd| d S )Nr   zindex.{})r   renamer    r!   r"   r   r   )r#   new_namer$   r$   r%   rename_index  s    z$RepositoryCheckTestCase.rename_indexc             C   s   t dd | jj D S )Nc             s   s   | ]}t |V  qd S )N)r   )r4   rB   r$   r$   r%   r     s    z7RepositoryCheckTestCase.list_objects.<locals>.<genexpr>)r   r   r]   )r#   r$   r$   r%   list_objects  s    z$RepositoryCheckTestCase.list_objectsc                s    j dddgddgdgg  jtddddddg j   jdd  jd  jt fd	d
  jj	   jdd  jdd  jddd  j
d  jdd  jtdddddg j  d S )Nr   r   r.   rq      r   T)r   c                  s
    j dS )Nr   )r   r$   )r#   r$   r%   rN     s    zGRepositoryCheckTestCase.test_repair_corrupted_segment.<locals>.<lambda>F)r   r   )r   rP   r   r   r   r   rQ   r   r   rW   r   )r#   r$   )r#   r%   test_repair_corrupted_segment  s     


z5RepositoryCheckTestCase.test_repair_corrupted_segmentc             C   s   | j dddgdddgg | jtddddddg| j  | jdd | jd | jj  | jddd	 | jtdddg| j  d S )
Nr   r   r.   rq   r   r   T)r   )r   r   )r   rP   r   r   r   r   r   rW   )r#   r$   r$   r%   test_repair_missing_segment  s     

z3RepositoryCheckTestCase.test_repair_missing_segmentc                sX    j dddgdddgg  jd  jtj fdd  jtdddg j  d S )	Nr   r   r.   rq   r   r   c                  s
    j dS )Nrq   )r   r$   )r#   r$   r%   rN     s    zLRepositoryCheckTestCase.test_repair_missing_commit_segment.<locals>.<lambda>)r   r   rQ   r   rR   rP   r   r   )r#   r$   )r#   r%   "test_repair_missing_commit_segment  s    
z:RepositoryCheckTestCase.test_repair_missing_commit_segmentc                s    j dddgdddgg ttjj jddd	d
d}|jdtj |jd W d Q R X  j	t
j fdd  jdd  jd  jtdddg j  d S )Nr   r   r.   rq   r   r   r   r   r   3zr+b   Xc                  s
    j dS )Nrq   )r   r$   )r#   r$   r%   rN     s    zNRepositoryCheckTestCase.test_repair_corrupted_commit_segment.<locals>.<lambda>T)r   r9   )r   r&   r   r    r!   r"   r   SEEK_ENDrz   rQ   r   rR   r   r   rP   r   r   )r#   r   r$   )r#   r%   $test_repair_corrupted_commit_segment  s    
z<RepositoryCheckTestCase.test_repair_corrupted_commit_segmentc                s    j dddgg ttjj jddddd}|jdtj |jd	 W d Q R X  j	t
j fd
d  jdd  jdd  j j dg  jddd  j j dg  jdd  jd  jtdddg j  d S )Nr   r   r.   r   r   r   1zr+br  c                  s
    j dS )Nrq   )r   r$   )r#   r$   r%   rN     s    z@RepositoryCheckTestCase.test_repair_no_commits.<locals>.<lambda>F)r   zindex.1T)r   r   zindex.3r9   )r   r&   r   r    r!   r"   r   r  rz   rQ   r   CheckNeededr   rP   r   r   r   r   )r#   r   r$   )r#   r%   test_repair_no_commits  s    
z.RepositoryCheckTestCase.test_repair_no_commitsc             C   s\   | j dddgdddgg | j  | jdd | jd | jtddddddg| j  d S )	Nr   r   r.   rq   r   r   T)r   )r   r   r   r   rP   r   r   )r#   r$   r$   r%   test_repair_missing_index  s
    
z1RepositoryCheckTestCase.test_repair_missing_indexc             C   s   | j dddgdddgg | j| j dg | jd | jd	d
 | j| j dg | jd | jtddddddg| j  d S )Nr   r   r.   rq   r   r   zindex.3z	index.100T)r   )r   rP   r   r   r   r   r   r   )r#   r$   r$   r%   test_repair_index_too_new  s    

z1RepositoryCheckTestCase.test_repair_index_too_newc             C   s   | j jtdd | j jtdd tjtd}| j j  |j  W d Q R X | j  | j * | j	dd | j
| j jtdd W d Q R X d S )Nr   s   datas   data2r   T)r   )r   r/   r   r   r   r   r0   r   r-   r   rP   rL   )r#   rl   r$   r$   r%   test_crash_before_compact%  s    
z1RepositoryCheckTestCase.test_crash_before_compactN)FT)rF   rG   rH   r   r   r   r   r   r   r   r   r   r   r   r   r   r  r  r  r	  r
  r  r$   r$   r$   r%   r     s&   
	
	r   c               @   s   e Zd Zdd ZdS )RepositoryHintsTestCasec             C   s   | j jtdd | j| j ji  t| j jdks6t| j jtd | j j	  | j jtdd | j j	  | j
td| j j | j
d| j j | j
d| j j d S )Nr   s   datar   )r   r/   r   rP   r   r^   rl   rd   r1   r0   Zassert_not_inr   )r#   r$   r$   r%   test_hints_behaviour4  s    

z,RepositoryHintsTestCase.test_hints_behaviourN)rF   rG   rH   r  r$   r$   r$   r%   r  2  s   r  c               @   sB   e Zd ZdZdddZdd Zdd Zd	d
 Zdd Zdd Z	dS )RemoteRepositoryTestCaseNFc             C   s"   t tdtjj| jd d|dS )Nz__testsuite__:r   T)r   r   )r
   r   r   r    r!   r"   )r#   r   r$   r$   r%   r&   H  s    zRemoteRepositoryTestCase.openc             C   s   G dd d}| S )Nc               @   s$   e Zd ZdZdZg ZdZdd ZdS )z9RemoteRepositoryTestCase._get_mock_args.<locals>.MockArgsborg?   Nc             S   s
   t | |S )N)hasattr)r#   itemr$   r$   r%   __contains__S  s    zFRemoteRepositoryTestCase._get_mock_args.<locals>.MockArgs.__contains__)rF   rG   rH   remote_pathumaskdebug_topicsrshr  r$   r$   r$   r%   MockArgsM  s
   r  r$   )r#   r  r$   r$   r%   _get_mock_argsL  s    
z'RemoteRepositoryTestCase._get_mock_argsc                s    j t fdd d S )Nc                  s    j jdi S )N__init__)r   callr$   )r#   r$   r%   rN   Z  s    z;RemoteRepositoryTestCase.test_invalid_rpc.<locals>.<lambda>)rQ   r   )r#   r$   )r#   r%   test_invalid_rpcY  s    z)RemoteRepositoryTestCase.test_invalid_rpcc          r   C   sl  d}y| j jdddi W nN tjk
rh } z0t|jdks@t|jd | j jjksXtW Y d d }~X nX y| j jdddi W nN tj	k
r } z0t|jdkst|jd | j jjkstW Y d d }~X nX y| j jdddi W nT tj
k
r: } z4t|jdkst|jd | j jjks*tW Y d d }~X nX y| j jddd	i W nL tk
r } z.t|jdkszt|jd |kstW Y d d }~X nX y| j jddd
i W nL tk
r } z.t|jdkst|jd dkstW Y d d }~X nX y| j jdddi W nh tjk
r } zHt|jdksDt|jd |ksXt|jd | j jjksrtW Y d d }~X nX y| j jdddi W nL tk
r } z.t|jdkst|jd |kstW Y d d }~X nX y| j jdddi W nh tjk
rf } zH|js t|j dks2t|jdksBtt|jdksVtW Y d d }~X nX d S )Nztest stringZinject_exceptionZkindDoesNotExistr   r   AlreadyExistsr  r   r   foorR   r   r   Zdividez6ZeroDivisionError: integer division or modulo by zero
ZeroDivisionError)r   r  r   r  r^   argsrd   locationZ	processedr  r  r   r   rR   r   r
   ZRPCErrorZunpackedZget_messageZexception_classZexception_full)r#   s1er$   r$   r%   test_rpc_exception_transport\  sX    **,&&,&z5RemoteRepositoryTestCase.test_rpc_exception_transportc             C   s   | j  }|| j_| jjtdddgks,t| jjtdddgksHt| jjtdddgksdt| jjtdddd	dgkstd
tjd< | jjtddddgkstd S )Nzexample.com:foosshzexample.comzssh://example.com/foozssh://user@example.com/foozuser@example.comzssh://user@example.com:1234/fooz-pZ1234z	ssh --fooZBORG_RSHz--foo)r  r   _argsssh_cmdr   rd   r   environ)r#   r!  r$   r$   r%   test_ssh_cmd  s     
z%RemoteRepositoryTestCase.test_ssh_cmdc             C   s8  | j jd ddtjdddgks"t| j }tj jtj	 | j j|dddddd	gksZtd
|_
| j j|ddd
ddd	gkstddg|_| j j|ddd
ddd	dgkst| j }d|_| j j|dddddd	gkstd|_| j j|dddddd	dgkstd|_|| j _| j jtdddddgks4td S )NT)Ztestingz-mzborg.archiverZserveFr  z--umask=077z--infozborg-0.28.2Zsomething_client_sideZrepository_compactionz.--debug-topic=borg.debug.repository_compactionr   iz--storage-quota=314159265z
ssh -i foozexample.com:foor&  z-ir  zexample.com)r   Zborg_cmdsys
executablerd   r  logging	getLoggersetLevelINFOr  r  r   r  r'  r(  r   )r#   r!  r$   r$   r%   test_borg_cmd  s$    "  
 z&RemoteRepositoryTestCase.test_borg_cmd)F)
rF   rG   rH   r   r&   r  r  r%  r*  r1  r$   r$   r$   r%   r  E  s   
6
r  c               @   s   e Zd ZdddZdd ZdS )RemoteLegacyFreeFc             C   s<   t jtdd$ ttdtjj| jd d|dS Q R X d S )NZ
dictFormatTz__testsuite__:r   )r   r   )r   r   r
   r   r   r    r!   r"   )r#   r   r$   r$   r%   r&     s    zRemoteLegacyFree.openc             C   s   | j jtdd | j j  | j j  | j | _ | j " | j jtdd | j j  W d Q R X | j | _ | j   | j jtd | j j  W d Q R X d S )Nr   s   foos   bar)r   r/   r   r0   r*   r&   r1   )r#   r$   r$   r%   test_legacy_free  s    



z!RemoteLegacyFree.test_legacy_freeN)F)rF   rG   rH   r&   r3  r$   r$   r$   r%   r2    s   
r2  c               @   s   e Zd ZdddZdd ZdS )RemoteRepositoryCheckTestCaseFc             C   s"   t tdtjj| jd d|dS )Nz__testsuite__:r   T)r   r   )r
   r   r   r    r!   r"   )r#   r   r$   r$   r%   r&     s    z"RemoteRepositoryCheckTestCase.openc             C   s   d S )Nr$   )r#   r$   r$   r%   r    s    z7RemoteRepositoryCheckTestCase.test_crash_before_compactN)F)rF   rG   rH   r&   r  r$   r$   r$   r%   r4    s   
r4  c               @   sL   e Zd Zdd Zdd Zdd Zdd Zd	d
 Zdd Zdd Z	dd Z
dS )RemoteLoggerTestCasec             C   s|   t j | _tj| j| _| jgtj jd d < g tjdjd d < g tjdjd d < tj	j
  tj	| _t j  | _	t_	d S )Nzborg.repositoryzborg.repository.foo)r;   StringIOstreamr-  StreamHandlerhandlerr.  handlersr+  stderrflush
old_stderr)r#   r$   r$   r%   r)     s    

zRemoteLoggerTestCase.setUpc             C   s   | j t_d S )N)r=  r+  r;  )r#   r$   r$   r%   r,     s    zRemoteLoggerTestCase.tearDownc             C   s0   t d | j| jj d | j| jj d d S )Nzunstructured stderr message
r3   z$Remote: unstructured stderr message
)r   rP   r7  getvaluer;  )r#   r$   r$   r%   test_stderr_messages  s    z)RemoteLoggerTestCase.test_stderr_messagesc             C   s0   t d | j| jj d | j| jj d d S )Nz%unstructured stderr progress messager3   z-Remote: unstructured stderr progress message)r   rP   r7  r>  r;  )r#   r$   r$   r%   test_stderr_progress_messages  s    z2RemoteLoggerTestCase.test_stderr_progress_messagesc             C   sN   | j jtj tj jtj td | j| jj d | j| j	j d d S )Nz,$LOG INFO Remote: borg < 1.1 format message
z"Remote: borg < 1.1 format message
r3   )
r9  r/  r-  DEBUGr.  r   rP   r7  r>  r;  )r#   r$   r$   r%   test_pre11_format_messages  s
    z/RemoteLoggerTestCase.test_pre11_format_messagesc             C   sN   | j jtj tj jtj td | j| jj d | j| j	j d d S )Nz=$LOG INFO borg.repository Remote: borg >= 1.1 format message
z#Remote: borg >= 1.1 format message
r3   )
r9  r/  r-  rA  r.  r   rP   r7  r>  r;  )r#   r$   r$   r%   test_post11_format_messages  s
    z0RemoteLoggerTestCase.test_post11_format_messagesc             C   sN   | j jtj tj jtj td | j| jj d | j| j	j d d S )Nz:$LOG INFO borg.repository Remote: new format info message
r3   )
r9  r/  r-  WARNINGr.  r   rP   r7  r>  r;  )r#   r$   r$   r%   test_remote_messages_screened  s
    z2RemoteLoggerTestCase.test_remote_messages_screenedc             C   s   t jdjt j t jdjt j | jjt j t j jt j tj }t j|}|jt j |gt jdj	d d < tj }t j|}|jt j |gt jdj	d d < t
d | j|j d | j|j d | j| jj d | j| jj d d S )Nzborg.repositoryzborg.repository.fooz;$LOG INFO borg.repository Remote: new format child message
r3   z!Remote: new format child message
)r-  r.  r/  r0  r9  rD  r;   r6  r8  r:  r   rP   r>  r7  r;  )r#   Zchild_streamZchild_handlerZ
foo_streamZfoo_handlerr$   r$   r%    test_info_to_correct_local_child  s"    

z5RemoteLoggerTestCase.test_info_to_correct_local_childN)rF   rG   rH   r)   r,   r?  r@  rB  rC  rE  rF  r$   r$   r$   r%   r5    s   	r5  )4r;   r-  r   r+   r+  r'   Zunittest.mockr   r   Z	hashindexr   Zhelpersr   r   r   Zlockingr   r	   Zremoter
   r   r   r   r   r   r   r   r   r   r   r   r   r3   r   r   r   r   r   rI   ri   r~   r   r   r   r   r   r   r  r  r2  r4  r5  r$   r$   r$   r%   <module>   sF   $+s9 &**  n