Linux

MySql 停止與 Laravel Queue、Supervisor 和 FFMPEg 結合執行

  • April 26, 2021

在設置隊列偵聽器以使用 FFMPEG 處理上傳的影片後,我多次返回伺服器發現 MySql 已停止執行。我檢查了驅動器空間,大約 77% 已使用(60G 中的 43G)。

這是我的程式碼,以防它有用:

public function fire($job, $data)
{
$data = json_decode($data['transcoding_message'], true);
$output_directory = '/home/ubuntu/transcodes/';
$amazon_array = array();

$s3 = AWS::get('s3');


//execute main transcoding thread

$cmd = 'sudo ffmpeg -i ' . $data['temp_file_url'] . ' -y -vcodec libx264 -tune zerolatency -movflags faststart -crf 20 -profile:v main -level:v 3.1 -acodec libfdk_aac -b:a 256k ' . $output_directory. $data['temp_file_key'] . '_HQ.mp4 -vcodec libx264 -s ' . $sq_width . 'x' . $sq_height . ' -tune zerolatency -movflags faststart -crf 25 -profile:v main -level:v 3.1 -acodec libfdk_aac -b:a 256k ' . $output_directory. $data['temp_file_key'] . '_SQ.mp4 -ss ' . $seek_half . ' -f image2 -vf scale=iw/2:-1 -vframes 1 ' . $output_directory. $data['temp_file_key'] . '_thumb.jpg';

exec($cmd." 2>&1", $out, $ret);


if ($ret)
{

   Log::error($cmd);
   echo 'Processing error' . PHP_EOL;
   //there was a problem
   return;
}

else
{
   //setup file urls
echo 'about to move files';

$hq_url = $this->bucket_root . $data['user_id'] . '/' . $data['temp_file_key'] . '_HQ.mp4';
$sq_url = $this->bucket_root . $data['user_id'] . '/' . $data['temp_file_key'] . '_SQ.mp4';;
$thumb_url = $this->bucket_root . $data['user_id'] . '/' . $data['temp_file_key'] . '_thumb.jpg';

$amazon_array['video_hq_url'] = $data['temp_file_key'] . '_HQ.mp4';
$amazon_array['video_sq_url'] = $data['temp_file_key'] . '_SQ.mp4';
$amazon_array['video_thumb_url'] = $data['temp_file_key'] . '_thumb.jpg';

       //copy from temp to permanent

foreach ($amazon_array as $k => $f)
{
   $uploader = UploadBuilder::newInstance()
       ->setClient($s3)
       ->setSource($output_directory.$f)
       ->setBucket($this->bucket)
       ->setKey('users/' . $data['user_id'] . '/' . $f)
       ->setConcurrency(10)
       ->setOption('ACL', 'public-read')
       ->build();

   $uploader->getEventDispatcher()->addListener(
       'multipart_upload.after_part_upload',
       function($event) use ($f) {
           // Do whatever you want
       }
   );

   try {
       $uploader->upload();
       echo "{$k} => Upload complete.\n" . PHP_EOL;

       DB::table('items')->where('id', $data['item_id'])->update(array($k => $this->bucket_root. $data['user_id'] . '/' .$f, 'deleted_at' => NULL));
       //delete local

       unlink($output_directory.$f);

       unset($uploader);
   } catch (MultipartUploadException $e) {
       $uploader->abort();
       echo "{$k} => Upload failed.\n" . PHP_EOL;
       continue;
   }
}

   //write to database

       DB::table('archives_items')->where('id', $data['archive_item_id'])->update(array('deleted_at' => NULL));
       DB::connection('mysql3')->table('video_processing')->where('id', $data['id'])->update(array('finished_processing' => 1));

   //delete files

   //delete s3
   $s3->deleteObject(
       array(
           'Bucket' => $this->temp_bucket,
           'Key' => $data['file_name']
       )
   );
   echo $data['temp_file_url'] . '=>' . " deleted from temp bucket.\n" . PHP_EOL;
   DB::connection('mysql3')->table('video_processing')->where('id', $data['id'])->update(array('deleted_at' => \Carbon\Carbon::now()));

}
   $job->delete();

   // end of processing uploaded video
   }


else
{
   return;
}

關於為什麼 MySql 會這樣死的任何想法?

ERROR 2002 (HY000): Can't connect to local MySQL server through socket     '/var/run/mysqld/mysqld.sock' (111)

我想提一下 php artisan queue:listen 命令是通過 Supervisor 觸發的,並且我有 4 個正在執行的並發程序。

錯誤日誌:

140613  5:37:52 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
140613  5:37:52 [Note] Plugin 'FEDERATED' is disabled.
140613  5:37:52 InnoDB: The InnoDB memory heap is disabled
140613  5:37:52 InnoDB: Mutexes and rw_locks use GCC atomic builtins
140613  5:37:52 InnoDB: Compressed tables use zlib 1.2.3.4
140613  5:37:52 InnoDB: Initializing buffer pool, size = 128.0M
140613  5:37:52 InnoDB: Completed initialization of buffer pool
140613  5:37:52 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
140613  5:37:52  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
140613  5:37:53  InnoDB: Waiting for the background threads to start
140613  5:37:54 InnoDB: 5.5.37 started; log sequence number 2922980986
140613  5:37:54 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
140613  5:37:54 [Note]   - '127.0.0.1' resolves to '127.0.0.1';
140613  5:37:54 [Note] Server socket created on IP: '127.0.0.1'.
140613  5:37:55 [Note] Event Scheduler: Loaded 0 events
140613  5:37:55 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.37-0ubuntu0.12.04.1'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  (Ubuntu)
140613  5:38:10 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
140613  5:38:10 [Note] Plugin 'FEDERATED' is disabled.
140613  5:38:10 InnoDB: The InnoDB memory heap is disabled
140613  5:38:10 InnoDB: Mutexes and rw_locks use GCC atomic builtins
140613  5:38:10 InnoDB: Compressed tables use zlib 1.2.3.4
140613  5:38:10 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
140613  5:38:10 InnoDB: Completed initialization of buffer pool
140613  5:38:10 InnoDB: Fatal error: cannot allocate memory for the buffer pool
140613  5:38:10 [ERROR] Plugin 'InnoDB' init function returned error.
140613  5:38:10 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
140613  5:38:10 [ERROR] Unknown/unsupported storage engine: InnoDB
140613  5:38:10 [ERROR] Aborting
140613  5:38:10 [Note] /usr/sbin/mysqld: Shutdown complete

InnoDB:mmap(137363456 字節)失敗;錯誤號 12

140613 5:38:10 InnoDB:致命錯誤:無法為緩衝池分配記憶體

您的系統記憶體不足錯誤 12 是核心信號 ENOMEM。

引用自:https://serverfault.com/questions/604997