How to create a new front-end Messenger Error type

• Check out the fer package:
  setup cvs -q online
  cvs checkout fer
  setup fer -r ~/fer -M ups -m fer.table
  • Do not forget the -M ups -m fer.table in the setup command!
  • Note that the above setup command assumes that you have checked out the fer package in your home directory. If you check it out elsewhere, replace ~/fer with the appropriate full directory path.
• Edit the fer/src/cdf_fer.msg file:
  • Decide which block in which to include the new error code
  • Blocks start with the indicator -b; each block used to have some specific meaning, but that meaning has been lost with time; choose -b 1 or -b 256 or -b 512 or -b 768; the most popular is -b 256
  • Do not exceed 256 message codes within one block!
  • The message code definition looks like this, for example:
    
    -m VME_BUS_ERROR -s E -t <%s>
    
    
    • The first argument after -m is the mnemonic abbreviation of the error
    • the -s argument indicates the severity, normally E for error or W for warning.
    • The -t argument shows the format of the error text. One may use simple C language printf style format specifiers. The easiest and most recommended format specifier is simply -t <%s> which allows any arbitrary string to be passed as an error message.
    • To guarantee that the message appears in the RunControl ErrorHandler, use the -s E error severity!
  • The mnemonic must be unique!
• Load the updated version of cdf_fer.msg into the merlin error code database:
  
  ssh b0gateway -l cdfprod
  setup fer devel
  cd $FER_DIR/src
  cp ~<username>/fer/src/cdf_fer.msg .
  setup merlin
  gmake cdf_fer_sysdb
  
  
  • Note that this procedure modifies files in the official merlin product area and so must be done from the cdfprod account. Replace the place holder <username> above with your own user name.
  • Alternatively, you may cvs commit the cdf_fer.msg changes and do a cvs update rather than the above cp copy command.
  • Watch carefully to make sure the process completes without errors; if there are errors, fix them right away.
  • This process will generate a new versions of the cdf_ferMsg.java and cdf_fer_msg_c.h files.
    • These files are used by Java and C language source files, respectively.
    • Note that the generated mnemonic tags will have FERML_ prepended to the mnemonic you provided in the original cdf_fer.msg file.
• Edit the fer/src/GENERAL.h file:
  • Add the same mnemonic from the cdf_fer.msg file to the list of either WARNING or ERROR messages.
  • Put the new code at the end of the WARNING or ERROR list, prepending WARNING_ or ERROR_ as appropriate.
  • Add a unique integer code for your new mnemonic.
  • Historical note: The GENERAL.h file was created before the merlin error code database was implemented, and thus represents an unfortunate redundancy in our system.
• Edit the FER_mess.c file:
  • Modify the FER_errorSender() or FER_warningSender() function, as appropriate.
  • Add a new case ... : statement in the main switch block of the function, using your WARNING_ or ERROR_ code from GENERAL.h as the argument of case.
  • A typical error error handler case in the FER_errrorSender() function looks like this:

          case (ERROR_VME_BUS_ERROR):
            err = MLencodedMessageCreate ((&errorText, FERML_VME_BUS_ERROR, 
    				       e.par7));
    	printf("*** %s ***\n", SHMEM_par7);
    	break;
    

  • The ERROR_VME_BUS_ERROR definition comes from GENERAL.h
  • The FERML_VME_BUS_ERROR definition comes from the generated cdf_fer_msg_c.h file.
  • The contents of SHMEM_par7 are printed here via the printf function for the convenience of the hardware debugger and code developer, and appear on the front-end console.
  • The parallel task code that calls FER_error(ERROR_VME_BUS_ERROR) must fill the shared memory variable SHMEM_par7 as appropriate (see next major item).
  • Other SHMEM variables are available for use for passing information into the arguments of the merlin format specifier, and are defined in the fer/src/FER.h file.
• Modify the code where the errors are being detected:
  • An example of using the VME_BUS_ERROR error code can be found in the FER_imp.c file, under the FER_attachInterrupt() function:

      intLevel = TRACER_getIntLevel();
      ...
      if (intLevel == RETURN_ERROR_READ) 
      {
        sprintf(SHMEM_par7,
        "FER_attachInterrupt: VME bus error slot=%d reading Tracer interrupt level [1]",
    	    VRClist->rh.tracerSlot);
        FER_error(ERROR_VME_BUS_ERROR);
        return RETURN_ERROR_ISRATTACH;
      }
    
    

  • Note that the SHMEM_par7 character string is filled with the details of the error that occurred.
  • The FER_error(ERROR_VME_BUS_ERROR) call here fills the Messenger task's error message queue with one new entry.
  • Error messages may be issued by any VxWorks task running on a front-end crate via this FER_error(...) call mechanism.
  • When the Messenger task is ready to read from the error message queue, it will pop the most recent message from the queue and send it via the merlin mechanism to the RunControl ErrorHandler.
  • Nota Bene: If the front-end processor is very busy, there may be a significant delay between a task adding a message to the queue, and the Messenger task reading and sending it over the network. For example, if trigger rates are very high, the Readout task may be dominating the front-end crate processor CPU, and the Messenger task may be unable to run, since the Messenger task has lower priority than the Readout task.
    • When considering modification of task priorities, use extreme caution, as improper task priorities can cause many subtle problems!
• Compile, test and commit changes.
  • To artificially generate a merlin error message code, there is a utility to do do:
    
    setup fer
    java rc.eh.MerlinSender
    
    
    • Select Add encoded message from the Add pull-down menu, then select the desider error mnemonic.
    • Be sure to set the partition before sending the message from the send option of the Send pull-down menu.
  • This program can be used to verify that the merlin message code was successfully added to the merlin database file.
  • During testing, you can use the vxreload command to load your private version of the fer product, for example:
    
    setup fer -r ~/fer -M ups -m fer.table
    vxboot b0ccal00
    ...wait for reboot to complete...
    vxreload b0ccal00
    
    
    • Note that a vxboot is always recommended immediately before doing the vxreload in order to terminate any tasks running that may have loaded the fer code.