Saya memiliki instance nginx yang diatur untuk mencatat akses ke /var/log/nginx/access.log dan kesalahan ke /var/log/nginx/errors.log, tetapi begitu logrotate berjalan setiap minggu, file tersebut dipindahkan ke * .log.1 dan file * .log baru akan dibuat, tetapi nginx terus log ke file log.1 alih-alih file .log baru (dan tidak ada yang di-gzip). Pertama kali saya perhatikan ini, sudah 3 minggu sejak rotasi log dan log semakin besar. Menjalankan kill -HUP `cat /run/nginx.pid`
membuat nginx mulai masuk ke tempat yang tepat lagi, tetapi masalahnya dimulai lagi minggu berikutnya.
Alasan yang lebih penting ini membuat frustrasi adalah karena saya memiliki log yang diatur untuk mengunggah ke Loggly melalui rsyslog, dan ketika nginx berhenti masuk ke file saya memiliki polling rsyslog, maka hal-hal berhenti mengunggah dan saya tidak mendapatkan peringatan.
Saya curiga ada hubungannya dengan me-restart nginx, atau memuat ulang konfigurasi, karena itu tidak dimulai sampai saya membuat perubahan konfigurasi dan memuat ulang konfigurasi dengan cara yang saya pikir normal. Saya mencoba menjalankan kill -USR1 `cat /run/nginx.pid`
tetapi file terus login ke lokasi yang salah sampai saya berlari kill -HUP `cat /run/nginx.pid`
, yang saya sudah tahu tidak menyelesaikan masalah.
Adakah yang tahu apa yang terjadi? Saya akui saya bukan ahli dalam administrasi logrotate atau nginx, tetapi Google saya gagal dalam hal ini.
Ini adalah skrip nginx logrotate saya, dan beri tahu saya jika ada hal lain yang ingin Anda lihat. Nginx.conf tidak memiliki keistimewaan di dalamnya terkait dengan logging, selain mendefinisikan lokasi output.
/var/log/nginx/*.log {
weekly
missingok
rotate 52
compress
delaycompress
notifempty
create 0640 www-data adm
sharedscripts
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi \
endscript
postrotate
[ -s /run/nginx.pid ] && kill -USR1 `cat /run/nginx.pid`
endscript
}
EDIT: Saya pikir saya menemukan masalah. Ini adalah output dari menjalankan logrotate dalam mode debug:
$ sudo logrotate --force -d /etc/logrotate.d/nginx
reading config file /etc/logrotate.d/nginx
Handling 1 logs
rotating pattern: /var/log/nginx/*.log forced from command line (52 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/nginx/access.log
log needs rotating
considering log /var/log/nginx/error.log
log needs rotating
rotating log /var/log/nginx/access.log, log->rotateCount is 52
dateext suffix '-20141023'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /var/log/nginx/access.log.1 does not exist
renaming /var/log/nginx/access.log.52.gz to /var/log/nginx/access.log.53.gz (rotatecount 52, logstart 1, i 52),
renaming /var/log/nginx/access.log.51.gz to /var/log/nginx/access.log.52.gz (rotatecount 52, logstart 1, i 51),
renaming /var/log/nginx/access.log.50.gz to /var/log/nginx/access.log.51.gz (rotatecount 52, logstart 1, i 50),
renaming /var/log/nginx/access.log.49.gz to /var/log/nginx/access.log.50.gz (rotatecount 52, logstart 1, i 49),
renaming /var/log/nginx/access.log.48.gz to /var/log/nginx/access.log.49.gz (rotatecount 52, logstart 1, i 48),
renaming /var/log/nginx/access.log.47.gz to /var/log/nginx/access.log.48.gz (rotatecount 52, logstart 1, i 47),
renaming /var/log/nginx/access.log.46.gz to /var/log/nginx/access.log.47.gz (rotatecount 52, logstart 1, i 46),
renaming /var/log/nginx/access.log.45.gz to /var/log/nginx/access.log.46.gz (rotatecount 52, logstart 1, i 45),
renaming /var/log/nginx/access.log.44.gz to /var/log/nginx/access.log.45.gz (rotatecount 52, logstart 1, i 44),
renaming /var/log/nginx/access.log.43.gz to /var/log/nginx/access.log.44.gz (rotatecount 52, logstart 1, i 43),
renaming /var/log/nginx/access.log.42.gz to /var/log/nginx/access.log.43.gz (rotatecount 52, logstart 1, i 42),
renaming /var/log/nginx/access.log.41.gz to /var/log/nginx/access.log.42.gz (rotatecount 52, logstart 1, i 41),
renaming /var/log/nginx/access.log.40.gz to /var/log/nginx/access.log.41.gz (rotatecount 52, logstart 1, i 40),
renaming /var/log/nginx/access.log.39.gz to /var/log/nginx/access.log.40.gz (rotatecount 52, logstart 1, i 39),
renaming /var/log/nginx/access.log.38.gz to /var/log/nginx/access.log.39.gz (rotatecount 52, logstart 1, i 38),
renaming /var/log/nginx/access.log.37.gz to /var/log/nginx/access.log.38.gz (rotatecount 52, logstart 1, i 37),
renaming /var/log/nginx/access.log.36.gz to /var/log/nginx/access.log.37.gz (rotatecount 52, logstart 1, i 36),
renaming /var/log/nginx/access.log.35.gz to /var/log/nginx/access.log.36.gz (rotatecount 52, logstart 1, i 35),
renaming /var/log/nginx/access.log.34.gz to /var/log/nginx/access.log.35.gz (rotatecount 52, logstart 1, i 34),
renaming /var/log/nginx/access.log.33.gz to /var/log/nginx/access.log.34.gz (rotatecount 52, logstart 1, i 33),
renaming /var/log/nginx/access.log.32.gz to /var/log/nginx/access.log.33.gz (rotatecount 52, logstart 1, i 32),
renaming /var/log/nginx/access.log.31.gz to /var/log/nginx/access.log.32.gz (rotatecount 52, logstart 1, i 31),
renaming /var/log/nginx/access.log.30.gz to /var/log/nginx/access.log.31.gz (rotatecount 52, logstart 1, i 30),
renaming /var/log/nginx/access.log.29.gz to /var/log/nginx/access.log.30.gz (rotatecount 52, logstart 1, i 29),
renaming /var/log/nginx/access.log.28.gz to /var/log/nginx/access.log.29.gz (rotatecount 52, logstart 1, i 28),
renaming /var/log/nginx/access.log.27.gz to /var/log/nginx/access.log.28.gz (rotatecount 52, logstart 1, i 27),
renaming /var/log/nginx/access.log.26.gz to /var/log/nginx/access.log.27.gz (rotatecount 52, logstart 1, i 26),
renaming /var/log/nginx/access.log.25.gz to /var/log/nginx/access.log.26.gz (rotatecount 52, logstart 1, i 25),
renaming /var/log/nginx/access.log.24.gz to /var/log/nginx/access.log.25.gz (rotatecount 52, logstart 1, i 24),
renaming /var/log/nginx/access.log.23.gz to /var/log/nginx/access.log.24.gz (rotatecount 52, logstart 1, i 23),
renaming /var/log/nginx/access.log.22.gz to /var/log/nginx/access.log.23.gz (rotatecount 52, logstart 1, i 22),
renaming /var/log/nginx/access.log.21.gz to /var/log/nginx/access.log.22.gz (rotatecount 52, logstart 1, i 21),
renaming /var/log/nginx/access.log.20.gz to /var/log/nginx/access.log.21.gz (rotatecount 52, logstart 1, i 20),
renaming /var/log/nginx/access.log.19.gz to /var/log/nginx/access.log.20.gz (rotatecount 52, logstart 1, i 19),
renaming /var/log/nginx/access.log.18.gz to /var/log/nginx/access.log.19.gz (rotatecount 52, logstart 1, i 18),
renaming /var/log/nginx/access.log.17.gz to /var/log/nginx/access.log.18.gz (rotatecount 52, logstart 1, i 17),
renaming /var/log/nginx/access.log.16.gz to /var/log/nginx/access.log.17.gz (rotatecount 52, logstart 1, i 16),
renaming /var/log/nginx/access.log.15.gz to /var/log/nginx/access.log.16.gz (rotatecount 52, logstart 1, i 15),
renaming /var/log/nginx/access.log.14.gz to /var/log/nginx/access.log.15.gz (rotatecount 52, logstart 1, i 14),
renaming /var/log/nginx/access.log.13.gz to /var/log/nginx/access.log.14.gz (rotatecount 52, logstart 1, i 13),
renaming /var/log/nginx/access.log.12.gz to /var/log/nginx/access.log.13.gz (rotatecount 52, logstart 1, i 12),
renaming /var/log/nginx/access.log.11.gz to /var/log/nginx/access.log.12.gz (rotatecount 52, logstart 1, i 11),
renaming /var/log/nginx/access.log.10.gz to /var/log/nginx/access.log.11.gz (rotatecount 52, logstart 1, i 10),
renaming /var/log/nginx/access.log.9.gz to /var/log/nginx/access.log.10.gz (rotatecount 52, logstart 1, i 9),
renaming /var/log/nginx/access.log.8.gz to /var/log/nginx/access.log.9.gz (rotatecount 52, logstart 1, i 8),
renaming /var/log/nginx/access.log.7.gz to /var/log/nginx/access.log.8.gz (rotatecount 52, logstart 1, i 7),
renaming /var/log/nginx/access.log.6.gz to /var/log/nginx/access.log.7.gz (rotatecount 52, logstart 1, i 6),
renaming /var/log/nginx/access.log.5.gz to /var/log/nginx/access.log.6.gz (rotatecount 52, logstart 1, i 5),
renaming /var/log/nginx/access.log.4.gz to /var/log/nginx/access.log.5.gz (rotatecount 52, logstart 1, i 4),
renaming /var/log/nginx/access.log.3.gz to /var/log/nginx/access.log.4.gz (rotatecount 52, logstart 1, i 3),
renaming /var/log/nginx/access.log.2.gz to /var/log/nginx/access.log.3.gz (rotatecount 52, logstart 1, i 2),
renaming /var/log/nginx/access.log.1.gz to /var/log/nginx/access.log.2.gz (rotatecount 52, logstart 1, i 1),
renaming /var/log/nginx/access.log.0.gz to /var/log/nginx/access.log.1.gz (rotatecount 52, logstart 1, i 0),
rotating log /var/log/nginx/error.log, log->rotateCount is 52
dateext suffix '-20141023'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /var/log/nginx/error.log.1 does not exist
renaming /var/log/nginx/error.log.52.gz to /var/log/nginx/error.log.53.gz (rotatecount 52, logstart 1, i 52),
renaming /var/log/nginx/error.log.51.gz to /var/log/nginx/error.log.52.gz (rotatecount 52, logstart 1, i 51),
renaming /var/log/nginx/error.log.50.gz to /var/log/nginx/error.log.51.gz (rotatecount 52, logstart 1, i 50),
renaming /var/log/nginx/error.log.49.gz to /var/log/nginx/error.log.50.gz (rotatecount 52, logstart 1, i 49),
renaming /var/log/nginx/error.log.48.gz to /var/log/nginx/error.log.49.gz (rotatecount 52, logstart 1, i 48),
renaming /var/log/nginx/error.log.47.gz to /var/log/nginx/error.log.48.gz (rotatecount 52, logstart 1, i 47),
renaming /var/log/nginx/error.log.46.gz to /var/log/nginx/error.log.47.gz (rotatecount 52, logstart 1, i 46),
renaming /var/log/nginx/error.log.45.gz to /var/log/nginx/error.log.46.gz (rotatecount 52, logstart 1, i 45),
renaming /var/log/nginx/error.log.44.gz to /var/log/nginx/error.log.45.gz (rotatecount 52, logstart 1, i 44),
renaming /var/log/nginx/error.log.43.gz to /var/log/nginx/error.log.44.gz (rotatecount 52, logstart 1, i 43),
renaming /var/log/nginx/error.log.42.gz to /var/log/nginx/error.log.43.gz (rotatecount 52, logstart 1, i 42),
renaming /var/log/nginx/error.log.41.gz to /var/log/nginx/error.log.42.gz (rotatecount 52, logstart 1, i 41),
renaming /var/log/nginx/error.log.40.gz to /var/log/nginx/error.log.41.gz (rotatecount 52, logstart 1, i 40),
renaming /var/log/nginx/error.log.39.gz to /var/log/nginx/error.log.40.gz (rotatecount 52, logstart 1, i 39),
renaming /var/log/nginx/error.log.38.gz to /var/log/nginx/error.log.39.gz (rotatecount 52, logstart 1, i 38),
renaming /var/log/nginx/error.log.37.gz to /var/log/nginx/error.log.38.gz (rotatecount 52, logstart 1, i 37),
renaming /var/log/nginx/error.log.36.gz to /var/log/nginx/error.log.37.gz (rotatecount 52, logstart 1, i 36),
renaming /var/log/nginx/error.log.35.gz to /var/log/nginx/error.log.36.gz (rotatecount 52, logstart 1, i 35),
renaming /var/log/nginx/error.log.34.gz to /var/log/nginx/error.log.35.gz (rotatecount 52, logstart 1, i 34),
renaming /var/log/nginx/error.log.33.gz to /var/log/nginx/error.log.34.gz (rotatecount 52, logstart 1, i 33),
renaming /var/log/nginx/error.log.32.gz to /var/log/nginx/error.log.33.gz (rotatecount 52, logstart 1, i 32),
renaming /var/log/nginx/error.log.31.gz to /var/log/nginx/error.log.32.gz (rotatecount 52, logstart 1, i 31),
renaming /var/log/nginx/error.log.30.gz to /var/log/nginx/error.log.31.gz (rotatecount 52, logstart 1, i 30),
renaming /var/log/nginx/error.log.29.gz to /var/log/nginx/error.log.30.gz (rotatecount 52, logstart 1, i 29),
renaming /var/log/nginx/error.log.28.gz to /var/log/nginx/error.log.29.gz (rotatecount 52, logstart 1, i 28),
renaming /var/log/nginx/error.log.27.gz to /var/log/nginx/error.log.28.gz (rotatecount 52, logstart 1, i 27),
renaming /var/log/nginx/error.log.26.gz to /var/log/nginx/error.log.27.gz (rotatecount 52, logstart 1, i 26),
renaming /var/log/nginx/error.log.25.gz to /var/log/nginx/error.log.26.gz (rotatecount 52, logstart 1, i 25),
renaming /var/log/nginx/error.log.24.gz to /var/log/nginx/error.log.25.gz (rotatecount 52, logstart 1, i 24),
renaming /var/log/nginx/error.log.23.gz to /var/log/nginx/error.log.24.gz (rotatecount 52, logstart 1, i 23),
renaming /var/log/nginx/error.log.22.gz to /var/log/nginx/error.log.23.gz (rotatecount 52, logstart 1, i 22),
renaming /var/log/nginx/error.log.21.gz to /var/log/nginx/error.log.22.gz (rotatecount 52, logstart 1, i 21),
renaming /var/log/nginx/error.log.20.gz to /var/log/nginx/error.log.21.gz (rotatecount 52, logstart 1, i 20),
renaming /var/log/nginx/error.log.19.gz to /var/log/nginx/error.log.20.gz (rotatecount 52, logstart 1, i 19),
renaming /var/log/nginx/error.log.18.gz to /var/log/nginx/error.log.19.gz (rotatecount 52, logstart 1, i 18),
renaming /var/log/nginx/error.log.17.gz to /var/log/nginx/error.log.18.gz (rotatecount 52, logstart 1, i 17),
renaming /var/log/nginx/error.log.16.gz to /var/log/nginx/error.log.17.gz (rotatecount 52, logstart 1, i 16),
renaming /var/log/nginx/error.log.15.gz to /var/log/nginx/error.log.16.gz (rotatecount 52, logstart 1, i 15),
renaming /var/log/nginx/error.log.14.gz to /var/log/nginx/error.log.15.gz (rotatecount 52, logstart 1, i 14),
renaming /var/log/nginx/error.log.13.gz to /var/log/nginx/error.log.14.gz (rotatecount 52, logstart 1, i 13),
renaming /var/log/nginx/error.log.12.gz to /var/log/nginx/error.log.13.gz (rotatecount 52, logstart 1, i 12),
renaming /var/log/nginx/error.log.11.gz to /var/log/nginx/error.log.12.gz (rotatecount 52, logstart 1, i 11),
renaming /var/log/nginx/error.log.10.gz to /var/log/nginx/error.log.11.gz (rotatecount 52, logstart 1, i 10),
renaming /var/log/nginx/error.log.9.gz to /var/log/nginx/error.log.10.gz (rotatecount 52, logstart 1, i 9),
renaming /var/log/nginx/error.log.8.gz to /var/log/nginx/error.log.9.gz (rotatecount 52, logstart 1, i 8),
renaming /var/log/nginx/error.log.7.gz to /var/log/nginx/error.log.8.gz (rotatecount 52, logstart 1, i 7),
renaming /var/log/nginx/error.log.6.gz to /var/log/nginx/error.log.7.gz (rotatecount 52, logstart 1, i 6),
renaming /var/log/nginx/error.log.5.gz to /var/log/nginx/error.log.6.gz (rotatecount 52, logstart 1, i 5),
renaming /var/log/nginx/error.log.4.gz to /var/log/nginx/error.log.5.gz (rotatecount 52, logstart 1, i 4),
renaming /var/log/nginx/error.log.3.gz to /var/log/nginx/error.log.4.gz (rotatecount 52, logstart 1, i 3),
renaming /var/log/nginx/error.log.2.gz to /var/log/nginx/error.log.3.gz (rotatecount 52, logstart 1, i 2),
renaming /var/log/nginx/error.log.1.gz to /var/log/nginx/error.log.2.gz (rotatecount 52, logstart 1, i 1),
renaming /var/log/nginx/error.log.0.gz to /var/log/nginx/error.log.1.gz (rotatecount 52, logstart 1, i 0),
running prerotate script
running script with arg /var/log/nginx/*.log : "
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi \
"
renaming /var/log/nginx/access.log to /var/log/nginx/access.log.1
creating new /var/log/nginx/access.log mode = 0640 uid = 33 gid = 4
renaming /var/log/nginx/error.log to /var/log/nginx/error.log.1
creating new /var/log/nginx/error.log mode = 0640 uid = 33 gid = 4
running postrotate script
running script with arg /var/log/nginx/*.log : "
[ -s /run/nginx.pid ] && kill -USR1 `cat /run/nginx.pid`
"
removing old log /var/log/nginx/access.log.53.gz
error: error opening /var/log/nginx/access.log.53.gz: No such file or directory
Namun, hanya ada arsip hingga sekitar * .log.8.gz, jadi logrotate gagal saat mencoba berinteraksi /var/log/nginx/access.log.53.gz
. Kenapa dia mencoba melakukan itu? Saya kira saya perlu menyentuh file palsu untuk mengisinya? Sepertinya ini salah.
weekly
frekuensi. Periksa apakah Anda memiliki masalah lain dengan nginx./var/lib/logrotate/status
). Menghapusnya dan mencoba lagi bisa memperbaiki ini, jika tidak, buka tiket.Jawaban:
Bah, akhirnya saya menemukan jawabannya setelah lama menggali. Masalahnya, dalam kasus saya bukankah logrotate gagal. Pesan kesalahan itu baik-baik saja, dan tidak benar-benar menghentikan logrotate. Masalahnya adalah nginx tidak melepaskan pegangan file ke file log setelah menerima
-USR1
sinyal darikill
. Singkatnya, alasan tidak memuat ulang file log adalah karena/var/log/nginx
folder itu tidak dimiliki oleh pengguna yang sama dengan proses pekerja nginx (dimiliki oleh www-data, berjalan di bawah web). Saya tidak tahu bagaimana itu berubah (mungkin karena server ini dibuat ulang baru-baru ini), tetapi mengubah folder untuk dimiliki oleh pengguna yang sama dengan proses pekerja nginx (dan memperbaiki file logrotate untuk membuat log baru sebagai web) memperbaiki masalah ini.sumber
service nginx reload
. Setelah itu log saya tiba.log
dan tidak.log.1
access.log
danerror.log
memiliki izin akses juga/var/log/nginx
direktori harus memiliki izin eksekusi.Saya memiliki masalah yang sama di Ubuntu 14.04, tetapi setelah membaca jawaban Anda, saya memeriksa bahwa nginx saya berjalan di bawah www-data, yang memiliki folder, jadi ...
Namun, saya telah menemukan bug ini , yang menunjukkan kesalahan pada bagian postrotate di /etc/logrotate.d/nginx.
Untuk memperbaikinya, Anda dapat berkomentar
dan gunakan salah satu opsi berikut (dari komentar di bug itu) sebagai gantinya:
sumber
Penyebab lain untuk masalah ini mungkin mengisi disk .
CATATAN: Ini bahkan dapat terjadi dengan log akses yang mengisi tiba-tiba! Apakah mungkin?
Untuk mereproduksi ini, Anda mungkin memiliki log akses Anda mengisi secara tiba-tiba dengan banyak permintaan, setelah
logrotate
diaktifkan akan dipindahkanaccess.log
keaccess.log.1
, tetapi akan gagal untuk mengompres file berikutnya, dan karena itu gagal mengirimUSR1
sinyal ke nginx.Bagaimana Anda memperbaiki masalahnya?
sumber