Linux

在復製到所有遠端伺服器之前驗證所有文件

  • February 20, 2018

我有一個由兩個劇本組成的 ansible 劇本,如下所示:

  • 在呼叫 Play2 之前,第一次播放會驗證我正在執行我的 ansible 的 localbox 上的所有內容。如果我在 Play1 中的驗證失敗,那麼我根本不想啟動 Play2。
  • 僅當 Play1 驗證成功時才會開始第二次播放,如果遠端伺服器不存在,則此播放會在遠端伺服器上創建一個目錄,然後將process.tar.gz文件複製到所有遠端伺服器上的特定目錄中。

以下是我的劇本:

---
- name: Play 1
 hosts: 127.0.0.1
 tasks:
     - name: check whether we have all the necessary files
       shell: "ls files/ | wc -l"
       retries: 10
       delay: 10
       register: number_of_files
       until: "number_of_files.stdout == '10'"

     - name: Total number of files
       debug: msg="Total files {{ number_of_files.stdout }}"

     - name: check whether all the files are generated within a minute difference
       shell: "find files/ -mmin -1 -type f -print | wc -l"
       register: file_time
       failed_when: "file_time.stdout != '10'"

     - name: Total number of files
       debug: msg="Total files {{ file_time }}"

     - name: check whether success file was generated
       shell: "ls files/SUCCESS | wc -l"
       register: success_file
       failed_when: "success_file.stdout != '1'"

     - name: Success file
       debug: msg="{{ success_file }}"

     - name: compress all the files in tar.gz
       shell: "rm process.tar.gz; tar -czvf process.tar.gz -C files . --exclude='SUCCESS'"

- name: Play 2
 hosts: ALL_HOSTS
 serial: "{{ serial }}"
 tasks:
     - name: creates directory
       file: path=/data/taks/files/ state=directory

     - name: copy and untar process.tar.gz file
       unarchive: src=process.tar.gz dest=/data/taks/files/

     - name: sleep for few seconds
       pause: seconds=20

我想看看是否有更好的方法來寫我上面的ansible?由於我最近開始使用 ansible,所以不確定我是否遵循所有最佳實踐。我正在執行 ansible 版本ansible 2.4.3.0。此外,我總是從 ansible 收到有關此任務的警告:

- name: compress all the files in tar.gz
 shell: "rm process.tar.gz; tar -czvf process.tar.gz -C files . --exclude='SUCCESS'"

這是警告:

[WARNING]: Consider using file module with state=absent rather than running rm
[WARNING]: Consider using unarchive module rather than running tar

更新:

下面是我的 ansible playbook 所在的路徑,僅從這個目錄中,我執行我的PLAYBOOK1. files目錄包含所有文件,包括我需要壓縮成 tar.gz 的文件SUCCESS

jenkins@machineA:~/jobs/processdata/workspace$ ls -lrth
total 145M
drwxr-xr-x 2 jenkins jenkins 4.0K Feb 19 17:36 files
-rw-r--r-- 1 jenkins root    1.6K Feb 19 19:32 PLAYBOOK1.yml
-rw-r--r-- 1 jenkins root    1.6K Feb 19 19:32 PLAYBOOK2.yml

這是我在執行PLAYBOOK1您的更改後遇到的錯誤:

TASK [Check all files] **********************************************************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "examined": 0, "failed_when_result": true, "files": [], "matched": 0, "msg": "/files was skipped as it does not seem to be a valid directory or it cannot be accessed\n"}

查看我的評論#

劇本1

- hosts: YOURHOST1
 remote_user: "YOURUSER"
 become: True

 tasks:

 #Instead using shell, use find module... It will check if there are 10 files created before 1 minute in folder, otherwise playbook fails
 - name: Check all files
   find: paths=/var/lib/jenkins/jobs/processdata/workspace/files
         file_type=file
         age=-1m
         age_stamp=mtime
   register: files
   failed_when: files.matched < 10

 - name: Number of files
   debug: msg="Total files {{ files.matched }}"

 #Check for SUCCESS file, I didn't add an action to delete SUCCESS file... Hope you have any other script to do it
 - name: Check Success File
   find: paths=/var/lib/jenkins/jobs/processdata/workspace/files
         file_type=file
         patterns=SUCCESS
   register: success_file
   failed_when: success_file.matched < 1

 - name: Success file
   debug: msg='Success file is {{ success_file.files.0.path }}'

 - name: Remove previous tarFile
   file: path=/tmp/process.tar.gz
         state=absent

 #I've found a way to make it work
 - name: compress all the files in tar.gz
   archive: path="{{ files_to_archive }}"
            dest=/tmp/test.tar.gz
            format=gz
   vars:
       files_to_archive: "{{ files.files | map(attribute='path') | list }}"

- include: PLAYBOOK2.yml 

Playbook2如果前面的操作之一失敗,Ansible 將停止並且不會執行任何其他操作。

- hosts: YOURHOST2
 remote_user: "YOURUSER"
 become: True

 tasks:

 - name: creates directory
   file: path=/data/taks/files/
         state=directory

 - name: Copy tar file
   copy: src=/tmp/test.tar.gz
         dest=/tmp/process.tar.gz
         owner=root
         group=root
         mode=0744

 - name: copy and untar process.tar.gz file
   unarchive: src=/tmp/process.tar.gz
              dest=/data/taks/files/

 - name: Remove SUCCESS File
   file: path=/data/taks/files/SUCCESS
         state=absent

#I don't know why you'll need this:
 - name: sleep for few seconds
   pause: seconds=20

請考慮 ansible 的功能之一是冪等性,您應該編寫腳本以執行並繼續之前的操作(如果它們在之前的啟動中失敗)。您和我的腳本都不是冪等的,因為您需要查找比一分鐘前更新的文件,因此在下次啟動期間(如果出現故障),您的腳本會丟棄早於一分鐘的文件。

SUCCESS 文件也是如此,如果您有一個繼續刪除它的平台/腳本,則 Ansible 腳本可能在執行時失敗。

引用自:https://serverfault.com/questions/897142