Friday 31 October 2008

Router Crash

The router failed. The secondary took over, but not all services are reachable from the outside world.

Update: the problem is solved. The resolution took very long. The network problem interfered with alert procedures. We are working to improve alerting in these cases.

Saturday 25 October 2008

New registrations will be delayed.

New Weblin registrations will be delayed for a while. All other users can continue.

Update: Registrations should arrive soon now. Operation of email delivery back to normal.

Thursday 23 October 2008

Network Problems

A network component shows errors, but not so much, that the fail over is activated. Random connection losses. We are working to identify the component.

Update: networking has been repaired.

Wednesday 22 October 2008

XMPP Server Failure [Update]

The XMPP cluster failed because of high load. The clients are trying to reconnect so the operation continues when the cluster is back.

Update: The cluster is back and running after 4 minutes.

MUC Server Failure

The Multi User Chat cluster partly failed and about half of the layered virtual world was offline.

Server has been restarted and the world is online again.

Tuesday 21 October 2008

Upgrading XMPP Cluster

A new XMPP cluster is installed to cope with the growing load. The cluster will be rebooted (possibly several times) during the installaton. Connection to chat servers might be interrupted until everything is up and running.

The cluster has been prepared in advance. But the switch might still be bumpy, because of size and version upgrades at the same time.

Friday 17 October 2008

XMPP Maintenance Restart

XMPP has been restarted as a safety measure.

Thursday 16 October 2008

XMPP Maintenance Restart

XMPP has been restarted as a safety measure.

Monday 13 October 2008

XMPP Maintenance Shutdown [Update]

To introduce a new database the XMPP service will be restarted at about 11:10pm UTC.

Update: The XMPP services are back and running.

Partial XMPP Maintenance Restart

Part of the XMPP cluster has been restarted as a safety measure.

Forum Down [Update]

Due to a server malfunction the forum is currently down. We work on it to repair the server.

Update: The forum service is back and running.

XMPP Maintenance Restart

XMPP has been restarted as a safety measure.

Saturday 11 October 2008

Partial XMPP Maintenance Restart

Part of the XMPP cluster has been restarted as a safety measure.

Comment: weblin is experiencing very high load, which is a good thing. Due to the recent events, we are very closely watching the XMPP condition and restart occasionally to avoid dangerous conditions. New (more and larger) servers will be online soon.

Friday 10 October 2008

XMPP Maintenance Restart

XMPP has been restarted as a safety measure.

Wednesday 1 October 2008

XMPP Service Restart

We needed to restart the XMPP service to start a new module to improve our services.