|
2004 CDF E-Log -- Owl shift. Tue Mar 9, 2004 |
| SciCo |
DAQ Ace |
Monitoring Ace |
CO |
(Operations Manager) |
| Rick Field |
Tom Schwarz |
Susana Cabrera |
Andrei Loginov |
Rob Harr |
Start of Shift Notes:  Doing quiet time calabrations and testing 3 trigger
tables. Expect shot set-up in about an hour.
Tue Mar 9 01:10:30
Run 179743
Terminated at 2004.03.09 01:09:51 - RunControl
Tue Mar 9 01:10:31
Run 179743
TERMINATE: PHYSICS_2_02[2,424,437] - Tom x2080
Tue Mar 9 01:11:55
After www server debacle most of evening shift yesterday and early owl shift are in day shift
feb 12 2004. no idea why.
of note. all calibs done except plug so far. still in quiet time
- Tom
-- Tue Mar 9 06:06:59 comment by...Rob Harr (operations manager) -- | The shift e-logs and silicon e-logs (and therefore possibly all other e-logs) are missing between Feb. 12 and about 23:00 on Mar. 8. Is it possible to restore the missing parts?????? |
Tue Mar 9 01:13:08
We have been "flying blind". The WEBserver
has been down. It just cam back up at
1:05am. - Rick Field
Tue Mar 9 01:14:46
Calorim QIE: run 179736
CLC QIE: run 179730
TOF QIE: run 179731
COT: run 179733
MUON: 179734
BSC QIE: 179737
TOF TAC: 179739
CEM LED: 179738
SMX QIE: 179741
SPIKE: 179742 L1 Rate 57.6 Hz
- Tom
Tue Mar 9 01:16:51
Run 179744
Activated at 2004.03.09 01:15:57 - RunControl
Tue Mar 9 01:16:52
Run 179744
ACTIVATE: Another Test of PHYSICS_2_02[2,424,437] - Tom x2080
Tue Mar 9 01:16:53
Run 179744
Terminated at 2004.03.09 01:16:38 - RunControl
Tue Mar 9 01:17:54
Run 179744
TERMINATE: PHYSICS_2_02[2,424,437] - Tom x2080
Tue Mar 9 01:20:18
We have made a sucessfully run using the trigger table PHYSICS_2_02[2,424,437].
It was just bad luck last Friday, because there was an additional problem at the same time. - Guillelmo/Guilles :: (run 179744)
Tue Mar 9 01:25:24
Run 179745
Activated at 2004.03.09 01:24:53 - RunControl
Tue Mar 9 01:25:25
Run 179745
ACTIVATE: TEST_ALPHA_CLUSTERING[5,438,403] - Tom x2080
Tue Mar 9 01:29:57
Run 179745
Terminated at 2004.03.09 01:29:34 - RunControl
Tue Mar 9 01:31:06
Run 179745
TERMINATE: TEST_ALPHA_CLUSTERING_NOSPIKES[5,438,403] Done. All seemed fine. - Tom x2080
Tue Mar 9 01:36:22
 | New PINK cell in ISL , as predicted by SI experts before
leaving: I_AVDD_B4W1L3
Current: 1085 Good: 1100-1125 |
- Susana.
Tue Mar 9 01:38:29
Run 179746
ACTIVATE: Plug Laser Calib. BOTHPLUGS_NORMAL - Tom x2080
Tue Mar 9 01:45:46
Run 179746
TERMINATE: Plug Calib BOTHPLUGS_NORMAL - Tom x2080
Tue Mar 9 01:48:26
Run 179747
ACTIVATE: Plug laser callib. BOTHPLUGS_MM. bad order. ending this. - Tom x2080
Tue Mar 9 01:48:27
Run 179747
TERMINATE: ending due to incorrect order. - Tom x2080
Tue Mar 9 01:50:50
Run 179748
ACTIVATE: Plug Laser Calib. BOTHPLUGS_MM_C - Tom x2080
Tue Mar 9 01:55:01
Run 179748
TERMINATE: Plug Laser Calib. BOTHPLUGS_MM_C - Tom x2080
Tue Mar 9 01:55:02
Run 179749
ACTIVATE: Plug Laser Calib. BOTHPLUGS_MM - Tom x2080
Tue Mar 9 01:56:33
Run 179749
TERMINATE: Plug Laser Calib. BOTHPLUGS_MM - Tom x2080
Tue Mar 9 02:04:51
Run 179750
Activated at 2004.03.09 02:03:52 - RunControl
Tue Mar 9 02:05:12
Run 179750
ACTIVATE: Taking Cosmic Data. COSMICS_ALL with COSMICS[12,391,403] - Tom x2080
Tue Mar 9 02:06:16
Table PHYSICS_TEST_2_02[11,437,438] could not be tested because it was not available to run
control.
- Tom
Tue Mar 9 02:07:47
All Calibrations performed between late evening shift 3/8/4 and owl shift 3/9/4. thanks to Alison and her amazing memory of calibrations w/out the www server. - Tom
Tue Mar 9 02:09:54
Run 179750
Terminated at 2004.03.09 02:09:39 - RunControl
Tue Mar 9 02:10:03
Run 179750
TERMINATE: ending run. - Tom x2080
Tue Mar 9 02:14:09
Run 179751
ACTIVATE: Plug Laser Calib. BOTHPLUGS_NORMAL. Note to myself: Plug Laser Calibration much more useful when laser actually activated. - Tom x2080
Tue Mar 9 02:17:34
I checked "CDF II Consumer Summary" page for two runs 179728, 179727:
179728 SiliMon 10136 PROBLEM (Errors)
No_clusters_detected_in_layer_No_ADC_counts_detected_in_layer_
179727 SiliMon 7482 PROBLEM (Errors)
No_clusters_detected_in_layer_No_ADC_counts_detected_in_layer_
179728 ObjectMon 5645 PROBLEM (Errors)
EM:OK;JET:OK;MET:OK;MUON:PROBLEM;SVT:UNDEFINEDSILICON:OK;TAU:OK;NoneTRACK:OK;
- Andrei
-- Tue Mar 9 02:29:01 comment by...Andrei -- for run 179726:
179726 ObjectMon 159671 BAD (Errors) EM:OK;JET:OK;MET:BAD;MUON PROBLEM;SVT:UNDEFINEDSILICON:BAD;TAU
OK;NoneTRACK:OK;
179726 SiliMon 148193 PROBLEM
(Errors)No_clusters_detected_in_layer_No_ADC_counts_detected_in_layer_
-- Tue Mar 9 02:29:31 comment by...Andrei -- stage0 is ok for all runs above.
Tue Mar 9 02:20:36
Run 179751
TERMINATE: Plug Laser Calib. BOTHPLUGS_NORMAL - Tom x2080
Tue Mar 9 02:22:56
Run 179752
ACTIVATE: Plug Laser Calib. BOTHPLUGS_MM_C - Tom x2080
Tue Mar 9 02:25:08
Run 179752
TERMINATE: Plug Laser Calib. BOTHPLUGS_MM_C - Tom x2080
Tue Mar 9 02:26:47
Run 179753
ACTIVATE: Plug Laser Calib. BOTHPLUGS_MM - Tom x2080
Tue Mar 9 02:28:59
Run 179753
TERMINATE: Plug Laser Calib. BOTHPLUGS_MM - Tom x2080
Tue Mar 9 02:35:12
Run 179754
Activated at 2004.03.09 02:33:00 - RunControl
Tue Mar 9 02:35:13
Run 179754
ACTIVATE: Taking Cosmics. COSMIC_ALL COSMICS[12,391,403] - Tom x2080
Tue Mar 9 02:39:04
Run 179754
Terminated at 2004.03.09 02:37:08 - RunControl
Tue Mar 9 02:39:05
Run 179754
TERMINATE: ending run. - Tom x2080
Tue Mar 9 02:42:55
Run 179755
Activated at 2004.03.09 02:42:11 - RunControl
Tue Mar 9 02:42:56
Run 179755
ACTIVATE: COSMICS-ALL COSMICS[12,391,403] - Tom x2080
Tue Mar 9 03:07:36
MCR is having a problem with the driver bias
supply for LRF2. Expert is working on it.
- Rick Field
Tue Mar 9 04:30:50
Tom cannot find trigger table PHYSICS_TEST_2_02
to run. Rob Harr instructed us to page
Jonathan Lewis which I did. - Rick Field
-- Tue Mar 9 05:03:02 comment by...jdl -- As usual, we forgot to set DAQVALID when fixing the Level 2 build after an initial failure. Done now.
Tue Mar 9 05:01:31
Run 179755
Terminated at 2004.03.09 05:01:00 - RunControl
Tue Mar 9 05:14:16
Run 179755
TERMINATE: all hell broke loose. power fermilab wide went out. - Tom x2080
Tue Mar 9 05:18:15
We had a power failure through all of Fermilab.
Except for the antiprotons... the store is
still there. The beam was off at the time
of the outage. When the power returned
the Silicon cooling alarm was going (along
with many other alarms). The ACE hit the
Silicon Rack Crash Buttton. I called Silicon
and we talked with Rainer and we paged
Andy (cooling) and he is on his way down.
I called Rob Harr and he just arrived. - Rick Field
-- Tue Mar 9 05:40:23 comment by...Rick Field -- Feeder 35 fault/site-wide glitch!
Tue Mar 9 05:25:49
Cleared Clock Errors. All 3 lights were off. - Tom
Tue Mar 9 06:08:17
The PE Pickup bipolar signal is gone on the oscilloscope in the clock crate. went through
procedure to start clock after power outage but didn't solve the problem. expert paged.
- Tom
Tue Mar 9 06:09:34
We have a clock problem. Problem with the PE pickup bipolar signal (no green line sync pulse). I paged the clock man. - Rick Field
-- Tue Mar 9 06:25:07 comment by...Rick Field -- I called Steve Hahn at home and he is working on the problem.
-- Tue Mar 9 07:24:52 comment by...Tom -- Both clock errors and sequencer errors had to be reset. all seems to work now.
Tue Mar 9 06:25:02
Silicon cooling and rack powere recovered. - Andy H.
Tue Mar 9 06:50:18
We had two hear-beat alarms with the TOF, and re-started the TOF
PC twice: at 00:15 and at ~6:30.
- Susana.
Tue Mar 9 06:53:27
CEM-CHA-WHA hear beat alarms: the hot towers are:
Phi-Wedge Eta-Towers
CEM east 5RL 8 & 9
CHA east 5R 4,5,6,7
WHA west 3R -8
Expert working on it. - Susana.
Tue Mar 9 07:07:19
BMU in yellow (WARNING) in the CDF GLOBAL ALARM.
All the detector is in software Over/Under Voltage.
- Susana.
-- Tue Mar 9 07:14:38 comment by...Susana. -- Following expert instruccions, I press again STANDBY
and the BMU comes back to the normal "standby" status
in terms of voltages and currents and green in the CDF
global alarm pannel.
Tue Mar 9 07:09:07
L3 does not make it past partitioning. goes red. cleaned up EVB and tried to restart proxy.
still doesn't work. trying one more time.
- Tom
-- Tue Mar 9 07:12:42 comment by...Tom -- L3 Paged.
Tue Mar 9 07:44:04
Run 179756
ACTIVATE: COT Calib. - Tom x2080
Tue Mar 9 07:44:05
Run 179756
TERMINATE: COT Calib. - Tom x2080
Tue Mar 9 07:46:07
Run 179757
ACTIVATE: SMX QIE Calib. - Tom x2080
Tue Mar 9 07:46:57
Apparently after power outage its' a good idea to take calibrations (COT and Shower first). so
on to that.
- Tom
Tue Mar 9 07:48:08
Run 179757
TERMINATE: SMX QIE Calib. - Tom x2080
Tue Mar 9 07:49:31
Run 179758
ACTIVATE: Cal QIE Calib. - Tom x2080
Tue Mar 9 07:49:43
Shift Summary: ** WEBserver Back Up **
Flying blind until around 1am. WEBserver is
now back but is is missing many days. I
had to "end shift" for Feb 12 and then start
the shift for today. The days in between are
missing??
** Quiet Time Calibrations **
Did quiet time calibrations.
** Three Trigger Table Tests **
We did trigger (1) and (3) but Tom
could not find trigger (2).
** Were Waiting for Shot Set-Up **
** Yellow ISL-SVX Chips on SvxMon **
As soon as the database came up the
silicon experts said that there really
was no proplem with the yellow chips
seen on SvxMon during the previous
shift. Apparently when the server went
down SvxMon did not work properly and
was accessing the wrong information.
The "yellow" chips should have been
"blue"??
** Runs 179726-179727-179728 **
Marked these runs as good.
** LRF2 Problem **
MCR is having a problem with the driver bias
supply for LRF2. Expert is working on it.
We are taking cosmics.
** Feeder 35 Fault/Site-Wide Glitch **
We had a power failure through all of Fermilab.
Except for the antiprotons... the store is still
there. The beam was off at the time of the outage.
When the power returned the Silicon cooling alarm was
going (along with many other alarms). The ACE hit the
Silicon Rack Crash Buttton. I called Silicon and
OPS Manager. Rainer, Andy, and Rob Harr have
arrived and we are working on recovering.
The solonoid went into a slow dump down to
zero and then was restarted by Cryo. The
solonoid is now back up and looks stable.
** Calabrations **
Need to do calabrations when able.
End of Shift Numbers
|
CDF Run II
Runs None
Delivered Luminosity zero
Acquired Luminosity zero
Efficiency 100%
|
- Rick Field
Tue Mar 9 07:50:37
Run 179758
TERMINATE: Calorim QIE Calib. - Tom x2080
-- Tue Mar 9 07:56:28 comment by...Andrei -- Cot calibration:
COT Calibration 07:42:13 on 2004-03-09
aveStart: 304.60
total numBad: 2113 - Too Many Bad channels
numBad cells: 2
numBad cables: 2
numBad slots: 17
aveDrift: 0.068
max number drifted in crate: 91
total numDrifted: 657
number of drifted cells 12
number of drifted cables 7
number of drifted slots 0
Too many bad channels in calibration.
experts have been notified.
Tue Mar 9 07:55:23
| Run Number |
Data Type |
Physics Table |
Begin Time |
End Time |
Live Time |
L1 Accepts |
L2 Accepts |
L3 Accepts |
Live Lumi, nb-1 |
GR |
SC |
RC |
|
179743
x2BE1F |
BEAM |
PHYSICS_2_02 [2,424,437] |
01:00:26 |
01:09:51 |
00:09:18 |
4,879 |
1,674 |
1,464 |
0.000 |
|
1 |
0 |
|
179744
x2BE20 |
BEAM |
PHYSICS_2_02 [2,424,437] |
01:15:57 |
01:16:38 |
00:00:35 |
314 |
106 |
96 |
0.000 |
|
1 |
0 |
|
179745
x2BE21 |
BEAM |
TEST_ALPHA_CLUSTERING_NOSPIKES [5,438,403] |
01:24:53 |
01:29:34 |
00:04:36 |
605 |
604 |
604 |
0.000 |
|
1 |
0 |
| Totals |
|
|
|
07:55:02 |
00:14:30 |
5,798 |
2,384 |
2,164 |
0.000 |
|
|
|
-End of Shift Report