Start of Shift Notes:  

Continuing Store 3277.  Taking good data.

at 12:08 SCPU_TRACER_EVENT_ID Error: 
Hardware EVB detected again a problem with data quality in  
SCPU b0eb16 (forwarded by FER crate COT_17).  
 

This problem presists every 10 minutes approximately.

TOF PC frozen, two trigger inhibits: IFIX:TOF HV

SMACS frozen on TOF PC.  Restarted PC.  All should be fine.

PHA Tower Plots(YMon #16 PHA) cold tower in the PHA: phi=18, eta=21

WHA Tower Plots(YMon#11 WHA): 2 "new" hot towers compared to reference plots: (eta=-6,phi=23), (eta=-7, phi=13) see previous shift for other plots of WHA(they are the same as those we have now)

CPR Wire Plots(YMon #113 CPR) look hotter than the reference plots.

XMon: the hottest trigger: 

 L3 80 EXO_TWO_TRACK_L1_SEVEN_TRK2_v2 7.53  

average x-sec:22.28 
expected x-sec:8.88±1.78

now it is
9.42 (standard deviations from the expected x-sec)
comparing to
7.53 before...

CER_SVXMON_HALT_RECOVER_RUN_ERROR: 
Stuck Cellid I/B5/W5/L1/C8-11 .  
AUTO HRR was issued 

CER_SVXMON_HALT_RECOVER_RUN_ERROR 
Stuck Cellid S/B1/W5/L4/C7-13 .  
AUTO HRR was issued 

 SCPU_TRACER_EVENT_ID Error 
 Hardware EVB has detected a problem with data quality in  
 SCPU b0eb16  
The errorlog says: 
(forwarded by FER crate COT_17).  
The window that popped up from run control says: 
(forwarded by FER crate WCAL_02). 
Automatic HRR.

SCPU_TRACER_EVENT_ID Error 
Hardware EVB has detected a problem with data quality in  
SCPU b0eb16  
Errorlog says: forwarded by FER crate COT_17 
Run Control says: forwarded by FER crate LEVEL1_CAL_01 
Automatic HRR 

SCPU_TRACER_EVENT_ID Error 
Hardware EVB has detected a problem with data quality in  
SCPU b0eb16 (forwarded by FER crate COT_17). (idem in the  
errorlog and in the run control) 
AUTO HRR

 SCPU_BAD_VRB_BYTE_COUNT Error 
 Hardware EVB has detected a problem with data quality in  
 SCPU b0eb11.  
 AUTO HRR

Plot of Abort Gap losses last 20 minutes. at 21kHz right now. Silicon put in standby after first dip downwards in abort gap.

SVX MON is not sending the heart beat to the consumer error interface in the last 20 minutes.
The CO person is going to re-start SVXMON consumer.

B0PAGC below 20KHz after scrapping.

In the time interval 4:18 - 5:20 the run 179726 has been halted 
because of the situation related high abort gap losses.  
At 5:20 it was recovered and run.

Trigger inhibit is set from IFIX:VME POWER for a few seconds.

Frequescy of AUTO HRR due to SCPU_TRACER_EVENT_ID Error 
1:11  + 
1:13  + 2 min  
1:19  + 7 min 
1:57  + 38 min  
2:10  + 13 min 
2:14  + 4  min 
2:31  + 27 min 
2:44  + 13 min   
3:15  + 31 min  
3:39  + 24 min 
3:42  +  3 min 
3:47  +  5 min 
4:06  +  59 min 
4:14  +  8 min 
4:15  +   1 min 
6:00  +   105 min  

*** Store 3277 Continuing *** 
Run 179726:  Taking good data. 

*** WHA Hot Towers *** 
WHA has two hot towers Phi-3-eta-8-R and Phi-10-eta-6-L.  
This causes the WHA Global Alarm and HW Summary to be  
yellow. This is a know problem since it appears in the  
day E-log (3-7-04). We are continuing to run with it 
yellow (as they did earlier). 

*** Abort Gap Losses (1st time) *** 
B0PAGC went over 18 kHZ (reached a high of about 18.5). 
I called MCR and they said they would make a few 
minor adjustments.  The lowered the vertical base 
by 0.0005 and B0PAGC came down to 16 kHz so we 
did not have to turn Silicon to "stand-by". 

*** Abort Gap Losses (2nd time) *** 
B0PAGC went over 18 kHZ. I called MCR and they said  
they would make a few minor adjustments.  However  
this time B0PAGC stayed at around 19 kHz for about  
15 min and then there was a sharp drop.  I decided  
to turn the Silicon to "stand-by".  TevMon was not  
red yet but things seemed erratic and I was afraid  
the sharp drop would be followed by an even sharper  
rise.  I called MCR and told them that I was going to  
"stand-by" mode and they said "good".  Right after 
we went to "stand-by" the abort gap losses went 
over 20 kHz and TeVMon went red.  There was a spike 
to 30 kHz (see the plot).  We went to "scraping" 
mode and waited.  B0PAGC stayed above 20 kHz for about 
45 min and then MCR called and said they were going  
to scrape (note that D0 was just then going to 
"stand-by").  We were, of course, already in 
"stand-by".  After the scraping the abort gap 
losses stabalized at around 9 kHz and we turned 
everything back on (after we got permission from  
Silicon).  Unfortunately we had over an hour of 
down time... but we protected the Silicon! 

*** Muon PC 2 Died Twice *** 
Tom had to go down and restart it. 

Runs                   179726
Delivered Luminosity   718.8 nb-1  
Acquired Luminosity    594.2 nb-1  
Efficiency             82.7