The Check Program March 4, 2003

The Check Program -- How do I know if I have met the Standard?

Perl Script check.pl
The perl script check.pl has been writeen with a minimum of libraries and should run without modification under unix and linux. I have verified it works with both SUN's and Red Hat 7.3 Linux. Perl is fast, in less than a second, a hundred files can be checked. Here are the steps you need to follow.

Download the program check.pl , the list of valid processors processors.names.long and the list of valid stack types stack.type.names.long. If your processing sequence doesn't fit, send me (Eric) a request and I'll add your proposal!
Edit the check.pl script for the following:

Edit #!/usr/global/bin/perl -Tw to fit the location of your compilier do a which perl to find it on your system!
Edit open ( PROCESSOR_FILE, "../perl/processor.names.long" ) so the program can find the location of the processor.names file
Edit open ( STACKS_FILE, "../perl/stack.type.names.long" ) as above

Make a list of files you want to check in the file junk and run the program!
ls *sgy > junk
~ekeyser/perl/check.pl > check.lst
Note: All of the programs and support files are located in the directory ../segy/perl/. Read the readme for some more clues.
Example Data -- 2D and 3D
In order to help write to this standard, I have prepared some dummy workstation segy data files in the directory located here. Feel free to download these data and examine them in detail. While the 3D is 11meg, the 2D lines are quite small. Look at these files in detail, if you use the EnCana STACKformat. You should see many of the additional fields (as lower case), some of them are Surface_elevation, Total static applied. If you are working with gather data, then use the EnCana GATHERS format If you don't trust the conversion program (or it doesn't seem to be working), use either of the templates and re load the data into SeisX.
 ls -al
-rw-rw-r--   1 ekeyser  expl      212560 Feb 20 15:08 F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.sgy
-rw-rw-r--   1 ekeyser  expl     11183440 Feb 20 15:08 P173286.ECC-MERG-033DM-1.3D-JENSENMERGE.test-istk.200302.gox.sgy
-rw-rw-r--   1 ekeyser  expl      267600 Feb 20 15:08 P34808.43461.2D-43461.f-tpna-freq-tx-mig.200402.es.sgy
Notice that I have changed the values of the time samples to protect the innocent. The correct numbers are, 0, 10, 20, ... 190. Here's how to run the check program:
 ls *sgy > junk
 ~ekeyser/perl/check.pl

 1  1  1 F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.sgy
 1  1  1 P173286.ECC-MERG-033DM-1.3D-JENSENMERGE.test-istk.200302.gox.sgy
 1  1  1 P34808.43461.2D-43461.f-tpna-freq-tx-mig.200402.es.sgy
No error messages, it's a good thing. The ones at the beginning are pass flags for a very lite check (only the reference number), a lite check (external file name) and the last is a heavy check for some stuff in the internals. Look at the check.pl program to determine what exactly is being checked.
Here is another check program that you may find useful. I have all my data loaders run this program before any data are attached to my projects. It will catch duplicate shot points, duplicate XY's etc. While SeisX 3.5 and 3.6 tolerate less than perfect data, I have found that SeisX 4.0 in its present state does not. We use this all the time and here's how:
~ekeyser/perl/spids.pl
  External_File_Name                                               Trace Spid/li Cdp/tr  Xcoord   Ycoord Trace Spid/li Cdp/tr  Xcoord   Ycoord  Cdp's Num_Spid Ratio
b F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.sgy     1    103      1   541025  7697814   653    429    653   534156  7704743    653    326   2.00
b P173286.ECC-MERG-033DM-1.3D-JENSENMERGE.test-istk.200302.gox.sgy    3D      1      1   380237  5472078          217    161   371611  5448262
b P34808.43461.2D-43461.f-tpna-freq-tx-mig.200402.es.sgy               1   1001      1   628208  6175919   825   1413    825   617936  6175970    825    412   2.00
The letter b signifies the data came from a Big Endian (SUN) machine, the letter l would mean an Intel machine. Again no error messages, a good thing. The ratio at the end of 2.0 means we have two traces per shot point. If this number is wonky, we probably have a problem

What about the SeisX stuff - What is it?
So you want all the good stuff. I have attemped to reverse engineer the SeisX files. 2D data are a piece of cake. Do it first. 3D is more of a challange. For example, what does CT1, 2, 3 mean and what is Rotation. Glad you asked. Here is another perl script I use to take files that are close to SeisX and make them SeisX. In other words, this script will compute and populate the binary and trace header with Peak, RMS and Average amplitudes, computes Rotation, CT flags etc. It will even put the proper suffix on the end (0.MIG.sgy). It's worked on about 500 files so far so I think it's ok. If you have a data set that doesn't work, let me know and I will fix it! You have to have the SeisX flag = 0 for it to change the internals. Here's how I run this script
ls *sgy > junk
~ekeyser/perl/encana2seisx.pl
Working on F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.sgy is in EnCana format, you are already flagged as SeisX, I'll just rename you
mv  F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.sgy to F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.MIG.0.sgy
Working on P173286.ECC-MERG-033DM-1.3D-JENSENMERGE.test-istk.200302.gox.sgy is in EnCana format, you are already flagged as SeisX, I'll just rename you
mv  P173286.ECC-MERG-033DM-1.3D-JENSENMERGE.test-istk.200302.gox.sgy to P173286.ECC-MERG-033DM-1.3D-JENSENMERGE.test-istk.200302.gox.STK.0.sgy
Working on P34808.43461.2D-43461.f-tpna-freq-tx-mig.200402.es.sgy is in EnCana format, you are already flagged as SeisX, I'll just rename you
mv  P34808.43461.2D-43461.f-tpna-freq-tx-mig.200402.es.sgy to P34808.43461.2D-43461.f-tpna-freq-tx-mig.200402.es.MIG.0.sgy
fini -- 
Since these data are flagged as SeisX all we see is a name change. Note that the perl script will automatically change the file to a MIG or a STK! I'll clear the SeisX flag, rename the file back to it's and re run the same script. Here's the results:
ls *sgy > junk
~ekeyser/perl/clear_seisx_flag.pl
 F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.MIG.0.sgy Changed SeisX flag from 91 to Zero
 mv F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.MIG.0.sgy F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.sgy

~ekeyser/perl/encana2seisx.pl
Working on F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.sgy is in EnCana format, I'll update to SeisX
2D data
Trace 500
mv  F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.sgy to F101692.ECC-NWT-2003-25.2D-2003-25.f-pr-fk-pr-mig.200304.gox.MIG.0.sgy
fini -- 
Here we have the script encana2seisx.pl that will take a EnCana format file and compute the additional SeisX required stuff, amplitudes, 3D corner points (CT1, 2, 3) etc.
How do I interpret the output?
Here is what the sample output looks like:
 1  1 P173286.JENSEN3D.JENSENMERGE3-D.test-stk.022003.gox.sgy meets the Lite and the heavy EnCana 2003 SEGY standard
 **WARNING**  Line a43565.01-KUG-09.f-fk-mig-103.022003.gox.sgy Geometry Flag <0> NOT 2d or 3d
 1  0 a43565.01-KUG-09.f-fk-mig-103.022003.gox.sgy meets the Lite and DOES NOT meet the heavy EnCana 2003 SEGY standard: Please check above warnings
 **WARNING**  Line a44267.MKD-5.u-fk-str.022003.gox.sgy Internal Name does not agree with external name
 1  0 a44267.MKD-5.u-fk-str.022003.gox.sgy meets the Lite and DOES NOT meet the heavy EnCana 2003 SEGY standard: Please check above
 **WARNING**  Line p172939.02-SBL-2.u-fk-str.022003.gox.sgy Geometry Flag <0> NOT 2d or 3d
 1  0 p172939.02-SBL-2.u-fk-str.022003.gox.sgy meets the Lite and DOES NOT meet the heavy EnCana 2003 SEGY standard: Please check above warnings
**WARNING**  Line a44263.MKD-2N.f-fk-fx-mig.022003.gox.sgy Internal Name does not agree with external name
 1  0 a44263.MKD-2N.f-fk-fx-mig.022003.gox.sgy meets the Lite and DOES NOT meet the Heavy EnCana 2003 SEGY standard: Please check above warnings
 **WARNING**  Line a44263.MKD-2N.f-fk-fx-str.022003.gox.sgy Internal Name does not agree with external name
fini -- 
The first file is a 3D and it passes all checks. Note that there are two levels of check, a Lite version and a Heavy version (Note the first two fields denote the Lite and Heavy check, a 1 is a pass and 0 is a fail). The lite versions only checks the external name and the data format byte position within the binary header. This should reject most files that are not segy files in the first place. The heavy version makes more serious checks of the internal.
The second line does not have the Geometry flag set (Flags if the data are 2D or 3D) and the internal name for MKD-5 does not agree with the external name. The internal name needs to be padded with nulls (00) to be correctly interpreted by SeisX!
Next step is to move the good lines out of the way:
sort -u < check.lst > junk
vi junk
Edit out all lines that do not pass the checks and move them out of the way:
awk '{printf("mv %s tmp/.\n", $3)}' junk > run
chmod +x run
./run
The next section will deal with how to correct the problems that still exist
Howto use SeisX to Correct the Binary Header?

Ahhh, good question. We can always as our friendly programmer to fix it but there is an easier way.
We can use the encana_v100_stack.fmt or the encana_v100_gathers.fmt format and SeisX, SeisWare and make the changes. Make sure you have write permission before you try this.
SeisX doesn't even beep to warn you if it cannot update the file

Howto Correct many files?
Here is another perl script fixsegy.pl that will correct data in the binary header. In the example above, the GEOMETRY flag had to be set to the number 2, the Central Meridian to -135 and the LINE_NAME and Reference_Number updated to reflect the file name. For good measure, I converted these fields to upper case. Here is the part of the perl program that need to be updated!
  $GEOMETRY = 2;
  $Central_meridian = -135;
  $LINE_NAME = uc ($External_LINE_NAME);
  $Reference_Number = uc ($External_reference_number);
For those of you that do now know perl, the $ indicates a variable and the uc is a function to convert to upper case. Simple, eh! Here is how to correct those ~160 bad files. Pssssst: Perl is a really slick way to produce the EnCana SEGY standard, you don't even need to be a programmer to do it!
ls *sgy > junk
/dat/photon/gek/perl/check/fixsegy.pl
a43565.01-KUG-09.f-fk-mig-100.022003.gox.sgy a43565 01-KUG-09
a43565.01-KUG-09.f-fk-mig-103.022003.gox.sgy a43565 01-KUG-09
a43565.01-KUG-09.f-fk-mig-106.022003.gox.sgy a43565 01-KUG-09
a43565.01-KUG-09.f-fk-mig-109.022003.gox.sgy a43565 01-KUG-09
...
It took perl less than two seconds to make all these corrections. Now lets check them again:
ls *sgy > junk
/dat/photon/gek/perl/check/check.pl > report.lst
Now the error messages are all gone!
more report.lst
 1  1 a44267.MKD-5.u-fk-fx-mig.022003.gox.sgy meets the Lite and the Heavy EnCana 2003 SEGY standard
 1  1 a44267.MKD-5.u-fk-fx-str.022003.gox.sgy meets the Lite and the Heavy EnCana 2003 SEGY standard
 1  1 a44267.MKD-5.u-fk-str.022003.gox.sgy meets the Lite and the Heavy EnCana 2003 SEGY standard
 1  1 p172939.02-SBL-2.f-fk-fx-mig.022003.gox.sgy meets the Lite and the Heavy EnCana 2003 SEGY standard
 1  1 p172939.02-SBL-2.f-fk-prmig-88.022003.gox.sgy meets the Lite and the Heavy EnCana 2003 SEGY standard
 1  1 p172939.02-SBL-2.f-fk-prmig-91.022003.gox.sgy meets the Lite and the Heavy EnCana 2003 SEGY standard
Everything passes, time to ftp the data to EnCana!

Howto Extract details from the Binary and EBCDIC headers
Yet another perl script, encana_dump.pl. This one will check to see if the EnCana flag has been set and if it has will print out the details of the EnCana Binary Header and the EnCana EBCDIC (ASCII) header. This file should be useful for Landmark data loaders and seismic interpreters. Seismic Processors can also use this file to check that the stuff in the binary header has been correctly set. Here is how to run the job.
ls *sgy > junk
/dat/photon/gek/perl/check/encana_dump.pl | more
Here is what the output looks like:
#############################################################
File Name = 7094.NC-VX-66X.es.072002.ustr.sgy NOT in EnCana internal format
#############################################################
File Name = 71949.AAIS-88-40.f-str.042003.acp.sgy is in EnCana internal format
############# BINARY HEADER VALUES ##########################
Reference_Number=71949 Num_data_traces=1 Num_auxiliary_traces=0 Sample_interval=2000 Sample_interval_original_field=0 Number_samples=1000
Number_samples_original=0, Format_code=1, CDP_Fold=1, Trace_sorting_code=2, Vertical_sum_code=0, Sweep_frequency_start=0
Sweep_frequency_end=0,Sweep_length=0,Sweep_type_code=0,Trace_number_of_sweep_channel=0,Sweep_trace_taper_length_at_start=0
Sweep_trace_taper_length_at_end=0,Taper_type=0,Correlated_data_traces=0,Binary_gain_recovered=0,Amplitude_recovery_method=0
Measurement_system=1,Impulse_signal_polarity=0,Vibratory_polarity_code=0,Number_of_traces_in_file=0,Station_interval=0
Mean_absolute_value=0, Domain_code=0, Time_first_sample=0, START_WINDOW=0, END_WINDOW=0, PEAK_VAL=38113, AVG_VAL=2555, RMS_VAL=3611, TRACE_SORT=0
SEIS_DATUM=1000.0,SEIS_REPLACEMENT_VELOCITY=2100.0,MAXIMUM_ENSEMBLE_NUMBER=0,Latitude_minimum=  0.0000,Longitude_minimum=  0.0000,Latitude_maximum=  0.
0000
Longitude_maximum=   0.0,Central_meridian=-135,UTM_zone=0,NE_corner_UTMX=   0.0,NE_corner_UTMY=   0.0,NW_corner_UTMX=   0.0,NW_corner_UTM=   0.0
SE_corner_UTMX=   0.0,SE_corner_UTMY=   0.0,SW_corner_UTMX=   0.0,SW_corner_UTMY=   0.0,3D_xline_START_X=0,3D_inline_START_Y=0,3D_time_START_Z=0
3D_xline_END_X=271, 3D_inline_END_Y=0, 3D_time_END_Z=2000000, AXIS_TRACE=1, AXIS_LINE=2, AXIS_TIME=3, LINE_NAME=AAIS-88-40, TOTAL_PHASE=   0.0
TOTAL_GAIN=   0.0, TOTAL_GAIN_EXP=   0.0, ROTATION=0, CT_1=0, CT_2=0, CT_3=0, GEOMETRY=2, SeisX_flag=91, Encana_flag=100
############# EBCDIC (ASCII) HEADER ##########################
C1 PROCESSED BY: ARCIS PROCESSING          042003TEMP                          
C2 REF#AAIS-88-40 71949             PROJECTION:T2 R21  CM:3   MEAS UNITS:METERS
C3 SAMPLE INTERVAL:2000  MS   SAMPLE/TRACE:1001                    NAD:27      
C4 SPREAD:1500------25--SP--25------1500  1..60..SP..61..120 TR    ATS:2.5     
C5 STATION INTERVAL:20         SOURCE INTERVAL:100             FOLD:12         
C6 SOURCE: DYNAMITE            PHASE CORRECTION APPLIED:0                      
C7 DATUM ELEVATION:1000                 REPLACEMENT VELOCITY:2100              
C8 BULK TIME 1ST TO ZERO0               END TIME:2000                          
C9 FILTERED STRUCTURE                                                          
C10ACCURATE PROCESSING INFORMATION:                                            
C11 VIDORA                                                                     
C121DEMUX, 2GEOMETRY ASSIGNMENT-TRACE EDITS                                    
C133GAIN CORRECTION AND AMPLITUDE RECOVERY                                     
C144SURFACE CONSISTENT MULTI-COMPONENT DECON (ZERO PHASE OUTPUT)               
C155WINDOW:400-800@225,1200-1900@1200,80 MSEC OPER,0.1% PW),                   
C167REF STATICS(DATUM=1000M, VR=2100 M/S, WX=762M/S)                           
C178RESID STATICS (400-1900@+/-36, 2 ITERATIONS)                               
C189VEL ANALYSIS-CONSTANT PERCENTAGE MOVEOUT COHERENCY SPECTRA                 
C1910NORMAL MOVEOUT CORRECTION                                                 
C2011MUTE 120/0 260/300 1200/850 2400/1700 M/MS                                
C2112STACK 1200%                                                               
C2213FILTER 10/15 - 80/100 HZ                                                  
C2314RELATIVE AMPLITUDE SCALING 0-400, 300-1400, 1300-2000 MSEC.     
...
Howto use Perl to change the Internal EnCana (SeisX) Line name - April 17, 2002
This Howto will demonstrate how to efficiently change the internal line of EnCana (SeisX) based on the external line name. Here is the routine fixname.pl I'll show the following three ways:

Use the first dot separated field of the external line name fixname.pl
Use the second dot separated field of the external line name fixname.pl -f2
The last two fields of the second dot separated external line name fixname.pl -f22
This routine has been very useful to remove one of the tedious steps to load SeisX. It eliminates manually typing the internal line name! Just run the perl script and in seconds you have corrected your segy data.
First make a file called junk containing a list of the lines.
ls *sgy > junk
head junk
f101678.ECC-NWT-2003-11.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy
f101678.ECC-NWT-2003-11.f-pr-fk-pr-mig.042003.gox.MIG.0.sgy
f101679.ECC-NWT-2003-12.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy
f101680.ECC-NWT-2003-13.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy
f101681.ECC-NWT-2003-14.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy
f101684.ECC-NWT-2003-17.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy
f101684.ECC-NWT-2003-17.f-pr-fk-pr-mig.042003.gox.MIG.0.sgy
f101685.ECC-NWT-2003-18.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy
f101686.ECC-NWT-2003-19.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy
/home/ekeyser/perl/fixname.pl -f2
f101678.ECC-NWT-2003-11.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-11 to ECC-NWT-2003-11
f101678.ECC-NWT-2003-11.f-pr-fk-pr-mig.042003.gox.MIG.0.sgy 2003-11 to ECC-NWT-2003-11
f101679.ECC-NWT-2003-12.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-12 to ECC-NWT-2003-12
f101680.ECC-NWT-2003-13.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-13 to ECC-NWT-2003-13
f101681.ECC-NWT-2003-14.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-14 to ECC-NWT-2003-14
f101684.ECC-NWT-2003-17.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-17 to ECC-NWT-2003-17
f101684.ECC-NWT-2003-17.f-pr-fk-pr-mig.042003.gox.MIG.0.sgy 2003-17 to ECC-NWT-2003-17
f101685.ECC-NWT-2003-18.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-18 to ECC-NWT-2003-18
f101686.ECC-NWT-2003-19.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-19 to ECC-NWT-2003-19
/home/ekeyser/perl/fixname.pl -f22
f101678.ECC-NWT-2003-11.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy ECC-NWT-2003-11 to 2003-11
f101678.ECC-NWT-2003-11.f-pr-fk-pr-mig.042003.gox.MIG.0.sgy ECC-NWT-2003-11 to 2003-11
f101679.ECC-NWT-2003-12.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy ECC-NWT-2003-12 to 2003-12
f101680.ECC-NWT-2003-13.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy ECC-NWT-2003-13 to 2003-13
f101681.ECC-NWT-2003-14.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy ECC-NWT-2003-14 to 2003-14
f101684.ECC-NWT-2003-17.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy ECC-NWT-2003-17 to 2003-17
f101684.ECC-NWT-2003-17.f-pr-fk-pr-mig.042003.gox.MIG.0.sgy ECC-NWT-2003-17 to 2003-17
f101685.ECC-NWT-2003-18.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy ECC-NWT-2003-18 to 2003-18
f101686.ECC-NWT-2003-19.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy ECC-NWT-2003-19 to 2003-19
/home/ekeyser/perl/fixname.pl     
f101678.ECC-NWT-2003-11.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-11 to f101678
f101678.ECC-NWT-2003-11.f-pr-fk-pr-mig.042003.gox.MIG.0.sgy 2003-11 to f101678
f101679.ECC-NWT-2003-12.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-12 to f101679
f101680.ECC-NWT-2003-13.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-13 to f101680
f101681.ECC-NWT-2003-14.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-14 to f101681
f101684.ECC-NWT-2003-17.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-17 to f101684
f101684.ECC-NWT-2003-17.f-pr-fk-pr-mig.042003.gox.MIG.0.sgy 2003-17 to f101684
f101685.ECC-NWT-2003-18.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-18 to f101685
f101686.ECC-NWT-2003-19.f-pr-fk-fx-mig.042003.gox.MIG.0.sgy 2003-19 to f101686
Fast and efficient!
Site Owner: Eric Keyser
Last Updated: Nov 23, 2004