在Docker Compose多容器应用程序中,如何防止Postgres运行以前发布的语句

我正在运行一个Scrapy项目,其中有一个将数据写入PostgreSQL数据库的Item Pipeline。 这是一个Docker Compose多容器应用程序,具有以下docker-compose.yml

 version: '3' services: privoxy: build: ./privoxy links: - tor tor: build: context: ./tor args: password: "foo" scraper: build: ./scraper environment: - http_proxy=http://privoxy:8118 - PGPASSWORD=iperpassword links: - tor - privoxy - db volumes: - scraper-data:/scraper/apkmirror_scraper/data - scraper-jobdir:/scraper/apkmirror_scraper/crawls db: image: postgres environment: - POSTGRES_PASSWORD=iperpassword volumes: - db-data:/var/lib/postgresql/data volumes: scraper-data: scraper-jobdir: db-data: 

我正在使用官方存储库( https://hub.docker.com/_/postgres/ )中的postgres镜像,并且在https:// docs之后将/var/lib/postgresql/data目录制作成了一个卷.docker.com / compose / compose-file /#volume-configuration-reference 。

问题是,当我启动应用程序时, db服务输出日志和错误有关我以前尝试的东西,而不是我现在要做的事情。 这就是我所看到的:

 apkmirrorscrapercompose_tor_1 is up-to-date apkmirrorscrapercompose_db_1 is up-to-date apkmirrorscrapercompose_privoxy_1 is up-to-date Recreating apkmirrorscrapercompose_scraper_1 Attaching to apkmirrorscrapercompose_tor_1, apkmirrorscrapercompose_db_1, apkmirrorscrapercompose_privoxy_1, apkmirrorscrapercompose_scraper_1 scraper_1 | List of databases scraper_1 | Name | Owner | Encoding | Collate | Ctype | Access privileges scraper_1 | -----------+----------+----------+------------+------------+----------------------- scraper_1 | postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 | scraper_1 | template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres + scraper_1 | | | | | | postgres=CTc/postgres scraper_1 | template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres + scraper_1 | | | | | | postgres=CTc/postgres scraper_1 | (3 rows) scraper_1 | scraper_1 | Postgres is up - executing command scraper_1 | Tor appears to be working. Proceeding with command... scraper_1 | 2017-06-30 12:32:59 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: apkmirror_scraper) scraper_1 | 2017-06-30 12:32:59 [scrapy.utils.log] INFO: Overridden settings: {'BOT_NAME': 'apkmirror_scraper', 'NEWSPIDER_MODULE': 'apkmirror_scraper.spiders', 'SPIDER_MODULES': ['apkmirror_scraper.spiders']} scraper_1 | 2017-06-30 12:32:59 [scrapy.middleware] INFO: Enabled extensions: scraper_1 | ['scrapy.extensions.corestats.CoreStats', scraper_1 | 'scrapy.extensions.telnet.TelnetConsole', scraper_1 | 'scrapy.extensions.memusage.MemoryUsage', scraper_1 | 'scrapy.extensions.closespider.CloseSpider', scraper_1 | 'scrapy.extensions.logstats.LogStats'] db_1 | The files belonging to this database system will be owned by user "postgres". db_1 | This user must also own the server process. db_1 | db_1 | The database cluster will be initialized with locale "en_US.utf8". db_1 | The default database encoding has accordingly been set to "UTF8". db_1 | The default text search configuration will be set to "english". db_1 | db_1 | Data page checksums are disabled. db_1 | db_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok db_1 | creating subdirectories ... ok db_1 | selecting default max_connections ... 100 db_1 | selecting default shared_buffers ... 128MB db_1 | selecting dynamic shared memory implementation ... posix db_1 | creating configuration files ... ok db_1 | running bootstrap script ... ok db_1 | performing post-bootstrap initialization ... ok db_1 | db_1 | WARNING: enabling "trust" authentication for local connections db_1 | You can change this by editing pg_hba.conf or using the option -A, or db_1 | --auth-local and --auth-host, the next time you run initdb. db_1 | syncing data to disk ... ok db_1 | db_1 | Success. You can now start the database server using: db_1 | db_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start db_1 | db_1 | waiting for server to start....LOG: database system was shut down at 2017-06-29 13:40:40 UTC db_1 | LOG: MultiXact member wraparound protections are now enabled db_1 | LOG: database system is ready to accept connections db_1 | LOG: autovacuum launcher started db_1 | done db_1 | server started db_1 | ALTER ROLE db_1 | db_1 | db_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/* db_1 | db_1 | LOG: received fast shutdown request db_1 | waiting for server to shut down....LOG: aborting any active transactions db_1 | LOG: autovacuum launcher shutting down db_1 | LOG: shutting down db_1 | LOG: database system is shut down db_1 | done db_1 | server stopped db_1 | db_1 | PostgreSQL init process complete; ready for start up. db_1 | db_1 | LOG: database system was shut down at 2017-06-29 13:40:41 UTC db_1 | LOG: MultiXact member wraparound protections are now enabled db_1 | LOG: database system is ready to accept connections db_1 | LOG: autovacuum launcher started db_1 | LOG: received smart shutdown request db_1 | LOG: autovacuum launcher shutting down db_1 | LOG: shutting down db_1 | LOG: database system is shut down db_1 | LOG: database system was shut down at 2017-06-29 13:41:53 UTC db_1 | LOG: MultiXact member wraparound protections are now enabled db_1 | LOG: database system is ready to accept connections db_1 | LOG: autovacuum launcher started db_1 | LOG: received smart shutdown request db_1 | LOG: autovacuum launcher shutting down db_1 | LOG: shutting down db_1 | LOG: database system is shut down db_1 | LOG: database system was shut down at 2017-06-29 13:43:22 UTC db_1 | LOG: MultiXact member wraparound protections are now enabled db_1 | LOG: database system is ready to accept connections db_1 | LOG: autovacuum launcher started db_1 | LOG: received smart shutdown request db_1 | LOG: autovacuum launcher shutting down db_1 | LOG: shutting down db_1 | LOG: database system is shut down db_1 | LOG: database system was shut down at 2017-06-29 13:52:02 UTC db_1 | LOG: MultiXact member wraparound protections are now enabled db_1 | LOG: database system is ready to accept connections db_1 | LOG: autovacuum launcher started db_1 | ERROR: column "developer" of relation "apks" does not exist at character 31 db_1 | STATEMENT: INSERT INTO apks (url, title, developer, app, version_name, version_code, architectures, package, apk_file_size, android_min_version, android_target_version, supported_dpis, md5_signature, time_uploaded, time_scraped, image_urls, images, file_urls, files, created_on, updated_on) VALUES ('http://www.apkmirror.com/apk/sony-mobile-communications/backup-restore-unknown-developer/backup-restore-unknown-developer-1-2-a-1-23-release/backup-restore-1-2-a-1-23-android-apk-download/', 'Backup & restore 1.2.A.1.23', 'Sony Mobile Communications', 'Backup & restore', '1.2.A.1.23', '2360343', ARRAY['arm'], 'com.sonymobile.synchub', 9541452, '6.0', '7.1', ARRAY['nodpi'], 'eda39fedfc33ea939a5bb4926b3980c1', '2017-04-25T16:30:00+00:00'::timestamptz, '2017-06-29T14:10:39+00:00'::timestamptz, ARRAY['http://img.gdocker.com/postgresql/ap_resize.png&w=96&h=96&q=100'], '{}', ARRAY['http://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=203068'], '{}', '2017-06-29T14:19:21.661193'::timestamp, '2017-06-29T14:19:21.661207'::timestamp) RETURNING apks.id db_1 | LOG: received smart shutdown request db_1 | LOG: autovacuum launcher shutting down db_1 | LOG: shutting down db_1 | LOG: database system is shut down db_1 | LOG: database system was shut down at 2017-06-29 16:28:54 UTC db_1 | LOG: MultiXact member wraparound protections are now enabled 

虽然我没有在应用程序上显示细节,但我的主要观点是,我确信错误消息中显示的数据来自上一次运行,而不是来自我正在尝试扫描的APK下载页面。

简而言之,PostgreSQL似乎是“caching”了以前的语句,甚至在我停止应用程序之后,试图重新运行它们,执行docker-compose builddocker-compose up 。 在我看来,这个行为必须以某种方式由/var/lib/postgresql/data ,因为这个卷是唯一一个在重新启动容器的过程中持久化的东西。

这确实是这样吗? 如果是这样,我该如何禁用这个“语句caching”的行为?

这只是意味着docker-compose正在重新启动您以前停止的容器(或者重新附加到它,如果它已经在运行),并且因为没有必要重新构build镜像:

 apkmirrorscrapercompose_db_1 is up-to-date 

所以,你在日志中看到的是“容器”的日志,其中包含了postgres的所有生命日志。

简单的删除容器,然后重新启动(db数据被保存在卷中,所以它将存活):

 docker-compose rm db