CERN batch announcements

ATLAS LSF groups re-ornganized and limit for local jobs introduced

ricardo — Thu, 07 Jul 2011 13:25:23 GMT

The ATLAS groups in LSFWeb were re-organized, all groups used for special purposes were moved out of u_ATLAS.

At the same time a limit of 100 running jobs in the shared resources was introduced for local job submissions.

More x-windows development packages available on batch

ricardo — Thu, 28 Apr 2011 06:22:16 GMT

A few extra -devel rpms where installed on batch nodes (most of the non-devel versions where already installed):

SDL
SDL-devel
Xaw3d
Xaw3d-devel
gd-devel
giflib-devel
libXScrnSaver-devel
libXTrap-devel
libXcomposite
libXcomposite-devel
libXdamage
libXdamage-devel
libXevie-devel
libXfont-devel
libXfontcache-devel
libXmu-devel
libXpm-devel
libXres-devel
libXtst-devel
libXv-devel
libXvMC
libXvMC-devel
libXxf86dga-devel
libXxf86misc-devel
libXxf86vm-devel
libdrm-devel
libfontenc-devel
libmng
libmng-devel
libtiff-devel
mesa-libGLU-devel
netpbm-devel
xorg-x11-xtrans-devel
xrestop
openmotif-devel

Relaxed limits for medium queues

ricardo — Mon, 28 Mar 2011 09:53:04 GMT

The job slot limits for the medium queues have been relaxed:

8nh    1000 -> 1500
8nh80  1200 -> 1750
1nd    1000 -> 1500
1nd80  1200 -> 1750
2nd    1000 -> 1250
2nd80  1100 -> 1500

As well as the general total job slot limit:

1500 -> 2000

New version of OpenAFS is being deployed in production

ricardo — Tue, 08 Mar 2011 09:16:35 GMT

The latest version of OpenAFS (1.4.14-0.cern) is being deployed in production on lxbatch. This version seems to address the slowdowns observed over the past month in lxbatch and lxplus.

At the same time a minor software upgrade is being deployed in preprod.

LSF 7.0.6-6 deployed in production on 11/11/2010

ricardo — Mon, 15 Nov 2010 08:04:02 GMT

LSF 7.0.6-6 deployed in production on 11/11/2010

This version includes the latest patches to address the issues with the mechanism that transfers files, used mainly by the Grid jobs.

All remaining CMS dedicated resources moved to SLC5

mccance — Thu, 11 Nov 2010 08:42:52 GMT

All nodes in cmsprco, cmsphedex, cmsrepack

lhcbinter resources moved to SLC5

mccance — Thu, 11 Nov 2010 08:41:52 GMT

All nodes now upgraded.

Batch service back after period of instability

mccance — Wed, 20 Oct 2010 11:48:50 GMT

There was an problem with the batch service this morning (05.30 until 10.49) resulting in degraded response from master batch daemon (which handles user requests). After an intervention from the vendor, the issue has been resolved. No jobs were lost.

The 'test' queue has been added to CERN lxbatch

mccance — Fri, 08 Oct 2010 14:33:58 GMT

We've just added a high-priority 'test' queue. It has a maximum CPU limit of 2 weeks allowing you to test any job you might like in it. The queue's priority is set very high so you should always be able to get your test job through provided you have some batch share left.

There is a limit of 1 running job per user for the queue.

The queue is different from the others in that you *must* specify the expected wall run time (or else it will default to 1 minute).

To specify this, please check the LSF bsub syntax for option '-W'. e.g.

bsub -W 20:00 -q test myjob.sh

which will request a runtime of 20 hours (currently this is real hours, not normalised to CPU power).

Specifying the expected run time will allow LSF to schedule your job more efficiently.

SLC4 batch capacity is back to normal

mccance — Fri, 01 Oct 2010 15:22:30 GMT

SLC4 batch capacity is back to normal. All nodes have been rebooted. Several long jobs had to be killed in order to recover the capacity in a timely manner.

35% of SLC4 batch capacity now back

mccance — Thu, 30 Sep 2010 20:43:30 GMT

35% of SLC4 batch resources (public and dedicated) are now patched and back in production. The remainder of the SLC4 capacity will be returned tomorrow morning - longer batch jobs running at this point.

SLC4 batch services are currently being drained for emergency reboot

mccance — Thu, 30 Sep 2010 15:05:37 GMT

SLC4 batch services are currently being drained to apply an emergency security patch which requires a node reboot. Some longer running batch jobs will be killed in order to recover the SLC4 capacity in a timely manner. The batch queues are currently accepting new SLC4 batch jobs, but not dispatching them to the worker nodes.

Public blog added

mccance — Wed, 29 Sep 2010 15:13:09 GMT

The public batch blog is now available on Feedburner and linked from the websites.

General job slot limits increased

ricardo — Mon, 23 Aug 2010 09:03:06 GMT

To reflect the increase in batch capacity, some of the job limits have been increased.

8nh            700 -> 900
1nd            750 -> 900
1nd80          800 -> 950
2nd            750 -> 900
2nd80          800 -> 950
per user       850 -> 1000
per grid user 2000 -> 2200

General job slot limits increased

ricardo — Tue, 08 Jun 2010 15:23:13 GMT

The general job slot limits were increased to reflect the latest increases in capacity.

Job limits per user per queue

1nh    500 -> 600
8nh    500 -> 600
1nd    500 -> 600
1nd80  500 -> 650
2nd    400 -> 500
2nd80   -  -> 550
1nw    100 -> 150
1nw80  100 -> 200
2nw    100 -> 150
2nw80  100 -> 200

General job limit per user
```
500 -> 700
```

Batch Intervention on Wednesday 10th March 2010

ricardo — Tue, 09 Mar 2010 15:23:45 GMT

There will be a batch service intervention at 10.00 on Wednesday 10th March.

We will update the LSF version on the batch service to a new patch version (7.0.5 -> 7.0.6). We will also reboot the master batch nodes to pick up a new kernel.

There will be a two short periods (up to 5 minutes) when no new jobs can be submitted. Running jobs will not be affected.