Tips and Tricks

The essential logfile in the ROTSE-III system is the system logfile /var/log/rotse.log. Use tail -f to watch the file go by, and less to explore it in more detail if the system has gone down.

When the system shuts down (either intentionally or unintentionally) all the daemons receive a SIG: 15, which is the standard Linux TERM signal. Seeing all the daemons report that they got SIG: 15 is normal. We will need to find out which daemon started the avalanche.

To isolate the offending daemon, start at the end of the logfile (in less, the > command goes to the end) and search backwards for which daemon timed out. This can be accomplished by doing a backwards search in less, with the ?timeout command. You will see output similar to the following:

...

Aug 12 21:14:50 rotse3a schierd: Non-0 when resp = @Status2RA0000, 0003, 0000 39B3

Aug 12 21:14:50 rotse3a schierd: error bit 3 set on axis 0

Aug 12 21:14:53 rotse3a schierd: @RecentFaults Axis 1 High Output 8/12/2002, 20:19:36.277;

Aug 12 21:14:53 rotse3a schierd: MOUNT_ERROR: Running error_recover()

Aug 12 21:14:53 rotse3a schierd: Brake on axis 0

Aug 12 21:14:53 rotse3a schierd: Drive amp disabled on axis 0

Aug 12 21:14:53 rotse3a schierd: Reset failed.

Aug 12 21:14:53 rotse3a schierd: unrecoverable error

Aug 12 21:14:53 rotse3a schierd: begin shutdown, errno: Interrupted system call

Aug 12 21:14:53 rotse3a schierd: Clearing mount command...

Aug 12 21:14:53 rotse3a schierd: Sending carriage return

Aug 12 21:14:53 rotse3a schierd: closing mtfd

Aug 12 21:14:53 rotse3a schierd: closing focfd

Aug 12 21:15:00 rotse3a rotsed: daemon timeout (15.0098 > 15.00) exceeded, schierd

Aug 12 21:15:00 rotse3a rotsed: obtain_status failed

...

In this case, rotsed is reporting that schierd, which controls the mount, is having a problem. The lines preceeding this timeout usually contain an error report from the daemon as it went down. In this case, the key lines are:

Aug 12 21:14:50 rotse3a schierd: error bit 3 set on axis 0

Aug 12 21:14:53 rotse3a schierd: @RecentFaults Axis 1 High Output 8/12/2002, 20:19:36.277;

This particular error will be discussed in Section 9.5.3.

If you are having problems with the camera, refer to Section 9.4 for specific information on the camera log file.