Static Files From Mirror Server

27 Jun 2008 14:23

Today I've configured the mirror server, to serve static files (static.wikidot.com) from the main server using Lighttpd.

The only thing to do is assign an IP address (preferably a portable one) and make a change in DNS settings.

Now static files are server with a proxy from lighttpd working on the main server, but we could change it later.

This thing closes work that needed to be done with the mirror server (as4). All we have to deal with is now magic file detection and ozone_session errors.

PS: we have two lighttpd configuration files now:

  • /etc/lighttpd.conf — for serving static files
  • /etc/lighttpd-mirror.conf — for acting as a live mirror.

After assigning an IP address to the static server, we may want to produce one lighttpd configuration based on IP (or host name), but it's not important now.

Comments: 2

Blog Theme

26 Jun 2008 12:59

I've managed to create a draft of blog theme for wikidot.
I've applied it to my dev blog to test it and show off.

It's based on this template by Open Source Web Design.

Comments: 2

Mirror Server

24 Jun 2008 12:55

Today I've (almost) managed to create a mirror server for wikidot.com service.

Features:

  • CentOS distribution
  • almost live Wikidot read only mirror
  • database is replicated from the original service in real time to this server
  • user uploaded files are replicated in real time to this server using FS mirror
  • avatars are to be mirrored with rsync every now and then
  • uses Portable IP address: 67.228.37.27
  • lighttpd serves ALL content with FastCGI PHP
  • database is read-only (as being replication slave)
  • CVS configured to use SSH keys (no password asking)
  • Wikidot PHP source mainly from the current production server
  • Improvements (from CVS): uploaded files served like in OpenSource version

Problems:

  • FS mirror is not 100% exact, it may not synchronize some (little fraction of) files every now and then, so we must rsync them additionally, to make sure nothing's lost
  • if you were logged in to Wikidot before, it'll complain about not being able to write to ozone_session (because it's read only)
  • Flickr Gallery not working and causing the whole page to display just nothing
  • magic file recognition not working (it may be a problem in PHP configuration or an extension):
PHP Warning:  finfo_open(): Failed to load magic database at '/usr/share/misc/magic'. in /var/www/www.wikidot.com/wikidot/php/utils/- on line 3
PHP Warning:  finfo_file(): supplied argument is not a valid file_info resource in /var/www/www.wikidot.com/wikidot/php/utils/- on line 4
PHP Warning:  finfo_close(): supplied argument is not a valid file_info resource in /var/www/www.wikidot.com/wikidot/php/utils/- on line 5

UPDATE: Flickr problem solution:

  1. yum install php-pear-HTTP-Request
  2. chgrp lighttpd /var/lib/php/session

Remember:

  • when switching to mirror, we must restart memcached (or force to invalidate every item in it)

Comments: 2

Lighttpd Serving Wikidot

12 Jun 2008 06:46

I work on the configuration of the Lighttpd webserver and Wikidot tweaks to eventually run Wikidot on Lighttpd.

The configuration of Lighttpd:

  • .htaccess rewritten to lighttpd.conf
  • all rules are rewrite-once (like RewriteRule [L] in Apache)
    • mixing rewrite-repeat with rewrite-once is tricky
  • standard lighttpd fastcgi php configuration
  • serving uploaded files through php program (no more flags/private)
  • non-existing avatars handled by 404 error handler on avatar directories

The tweaks in Wikidot:

  • sanitized PrivateFileFlowController
  • created new: UploadedFileFlowController
  • they can eventually become one file (PrivateFileFlowController shouldn't be needed anymore)
  • created new: local.php (to substitute private_file_filter.php)
  • SSL seem to work correctly

Still to do:

  • replace (configure the backend) readfile() with Header("X-Sendfile…")
  • UploadedFileFlowController should deal with:
    • redirections
    • html files
    • secure *.wdupload.wikidotsyndication.com domain
    • SSL
    • magic database better

UPDATE: 2008-06-13: done some things, still to do:

  • replace (configure the backend) readfile() with Header("X-Sendfile…")
  • SSL is not ready, because $_SERVER["HTTPS"] is not supported in fast-cgi

PROPER FLOW:

if HOST = *.wdupload.wikidotsyndication.com:
    URL = http[s]://[site].wdupload.wikidotsyndication.com/files--upload/[file]
    send = 1
else:
    URL = http[s]://[domain]/files--upload/[file]
    if file is text/html or xml+html:
        301 to http[s]://[site].wdupload.wikidotsyndication.com/files--upload/[file]
    else
        send=1
if send:
    check permissions
    set mime type
    send file

Comments: 1

page 1 of 212next »
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License