ipchkmail

This program spools spool/xsmtp for files in MIME format left by
Sendmail/sffmail.

If can be used to :
	- check the Sender is valid for sending to that destination
		by checking against a list of names in tables/mail/check.
	- zap or reroute any mail without either a 'From' or a 'Reply-to' field.
	- check and/or split off and/or convert attachments

Normal running is to force all attachments into files and route them accoring
to their type - xls, jpeg, xml etc

It uses a parameter file, by default tables/mail/CHKMAIL. The format of this
file is :
	; comment line
	strip-mime-header: Strip the Mime Header of incoming files. default: no
		strip-mime-header:yes
		strip-mime-header:all	these two will always strip all mimes
		strip-mime-header:no	this will leave all mime lines
		strip-mime-header:some	this will check for 'ignore' see below
	defdest: (default Fip Destination - DU progs.FipHdr field)	default: FIPMSG
	chrset: (Source character set)				default: email
		ie this provides a different progs.FipHdr SC field to SC:email
	extra-fiphdr: (Extra Fip Hdr information to add to each file) def: none
	no-from-message:
	invalid-sender-message:
	check-sender: if there is a file in tables/mail/check, make sure the
		'From' or 'Reply-to' address is included.
	fiphdr: (Map incoming mail field to a Fip Hdr field)	default: none
		fiphdr:DI	Subject
		fiphdr:AZ	Message-Id
	convert-encoded-words: (8bit/utf8/no)		default: utf8
		Mime fields such as Subject are often in a format called 'encoded words'
			=?iso-8859-1?q?this=20is=20some=20text?=
		The DATA of the encoded word will usually be converted to UTF8 or 8bit
	ignore: (Mail header field to always ignore)	default: none
		Used with 'strip-mime-header:some' (see above) to filter
		out or delete selected mime header lines.
		ignore: Status
		ignore: Apparently-To
		ignore: X-Msmail-Priority
	fiphdr-TO-field: (progs.FipHdr) Use this where you already have the real TO
		address in a fipHdr field. eg if you are running sffmail in
		/etc/aliases :	 donkeyrides: "|/fip/bin/sffmail -hJA:DKR"
		then use 	fiphdr-TO-field:JA
		to capture all DKR traffic no matter what is in the To field.
	max-TO-length: Set the Maximum length of the 'To:' field to this.
		default: 400 chrs.
		Listservers often give tens/hundreds of names in the 'To:' field. This chops
to the first few.
	send-all-with-attachments: (DU)
	queue-all-with-attachments: (folder name)
		Fip Destination and folder name to send any file (complete,
		unaltered) with one or more attachments.
		If there is no "queue-..", the default output queue is used.
		This is incompatible with decode, -a/-A and any other
		attachment processing.
	decode:
	unencode: Decode any base64 sections.	default: no
		decode:		This will NOT decode attachments
		decode:text	This will NOT decode attachments
		decode:all	This will decode body AND attachments	
		A second parameter may be added 'utf8' which will decode the result to UTF8
			decode:text,utf8
	split-every-attachment: (yes/no)
		Normally any attachment that is has a Content-Type of 'text' is left in
		the main message. Use this keyword to split ALL attachments and either
		leave in the folder (-a) or sent to the destination (-A).
	att-message: Message to replace an attachment with.
		The attachment is called fipHdr DN.
		att-message:<a href="readdisboyo.pl?namo=\\DN>Read this Attachment
\\DN</a><br>
		default:** Attachment has been Stripped, pls contact System Managers for :
\\DN
	alt-html-inline: (yes/no)
		show the Alternate HTML version inline or NOT	default: yes
	add-fiphdr-on-atts:yes/no
		add the whole fiphdr on each attachment (-A only)
	att-data-only:
		do NOT add any mimeheaders on the top of the attachment (-A only)
	fiphdr-att-type: (XX)
	fiphdr-att-chrset: (XX)
	fiphdr-att-name: (XX)
		2 letter progs.FipHdr fields for the Content-Type, Charset and Filename of any
attachments
		no defaults
	start-att-marker: (progs.FipSeq)
		Extra Data to insert into the main data file at the point where an
attachement starts
		extra progs.FipHdr fields at this point are  PL : level, PI : inline YES or NO
			Plus the 3 fiphdr-att-* fields if specified
		option:VY	PI,,,,YES
		start-att-marker:\VYInLine attachment starts here ->\n\$o
		no defaults
	end-att-marker: (progs.FipSeq)
		Extra Data to insert into the main data file at the point where an
attachement ends
		PL : level, PI : inline YES or NO or IGNORE (not inline and no file)
		option:VN	PI,,,,NO
		end-att-marker:\VNEndOfStrippedAttachment\n\$o
		no defaults
	before: (progs.FipSeq)
	after: (progs.FipSeq)
		Extra Data to insert into the main data file at the top or tail
		no defaults

	preserve-existing-filename:
	preserve-existing-source:
	ignore-existing-filename:
	ignore-existing-source:
		These two will ignore either the progs.FipHdr field SN or SU for name or source
from the raw file and add new ones.
		The default is to preserve the source but ignore the filename!

	process-if-no-check-file:yes/no
		If you are using Check files and there is no check file for that
		address, the default is to SEND. Use the following to NOT send:
			process-if-no-check-file:no
	newDA: progs.FipHdr field to use in place of DA for the 'To:' field
		If you are using smtpwire, DA will already be filled in correctly.
		So map it to another fipHdr	newDA:RA
	newDI: progs.FipHdr field to use in place of DI for the 'Subject:' field
		If you are erworking the Subject line - forcing it to progs.QuotePrintable for
example,
		map it to another fipHdr	newDI:RI

-- Check files --
There needs to be a check file for EACH address you want checking. The filename
is just the NAME (not the Domain) forced uppercase :
	eg	if the TO address is	billy@never.inhale.com
		the check file is	'tables/mail/check/BILLY'

The entries in the check file are the sender's address. ie the FROM field.

The format of a check file is
	; is a comment line
	; there can be one or many entries, each on a separate line
	; for a single sender
	name@domain
	billyGoat@bridge.to.far.com
	; for a complete domain - prefix a '*'
	*@completely_idiotic.com

For Example
;
; Control files to progs.NationalDesk
;
; allow all internal users
*@tuebrook-bugle.com
; Star reporter
famousJock@ego.inflate.com
; the editor
S.Leaze@weasel.org

-- Blacklists --

A single BLACKLIST file can be added to ignore files FROM certain addresses
and/or domains.

It is the same syntax as the Check files (see above) but is in the main folder
tables/mail/BLACKLIST.

Note that you can block all domains for a name with a wild card too :

Undisclosed.Recipients@*

-- Conversions --

; What conversions do we want to do (double quotes to keep the spaces)
convert:binhex script:"/usr/forum/bin/alchemy \QN \QN.nohex -Q -o -e --."

-- progs.FipHdr fields --

Note the following progs.FipHdr fields are filled in for each incoming file (in
addition to any 'fiphdr' parameters)
	SA - From address
	DA - To address (can be changed using the 'newDA' parameter)
	DI - Subject field (can be changed using the 'newDI' parameter)
If the contents of these fields are a different charset, it is ignoted and
converted to the Binary chrs
However the 'fiphdr:' fields are preserved without conversion.

Note that progs.FipHdr fields are used for conversions:
	QN - Full Path and Filename of the input file to be converted
	QR - Path or queue for converted files and files to convert
		which defaults to /fip/spool/epsf
	SN - name of the input file to the program !
	QI - name of the scrolled queue


Input parameters are (all optional) :
	-a : Folder to put attachments we have stripped
						default: att left in data
	-A : Destination (progs.FipHdr DU) to send all the
		attachments we have stripped.	default: att left in data
	-i : input queue to scan		default: spool/xsmtp
	-o : output queue for processed files	default: spool/2go
	-d : done queue for incoming		default: none
		this will leave the raw, incoming file in this done folder.
	-l : do NOT log incoming files		default: log
	-w : file wait interval in seconds	default: no wait
		set this for nfs or slow systems eg -w 10
	-z : name of parameter file if not	default: tables/mail/CHKMAIL
		the default.
	-T : do NOT attempt to clean up the DU	default: do
		only letters, numbers '_' and '-' are allowed.
	-v : print version no and exit

Version Control
;001w17	20jun03 added fiphdr-to-field, process-if-no-check-file
	;b 26jul03 bugette - dbl ATT on attachment
	;c-g 30jul03 bugette - decode NOT always working correctly plus newDA added
	;h 31oct03 timings
	;i-j 05dec03 added add-fiphdr-on-atts
	;k-m 30mar04 bugette when hacker sends > 64000 on one line of header
	;n-o 03sep04 added check_for_fiphdr for attachments too
	;p-q 20sep04 speedy
	;r 09feb05 Imail has invalid Content-type
	;s-t 31oct05 bugette - need to be able to handle blank lines with spaces in
the mime header
	;u-v1 21mar07 bugette with attachments and long TO: addresses
	;w1-9 10dec08 attachments inside attachments - major mods ! added -1 for
single ;7 bugette without -a -A ;8-9 newDI added
		;10-11 added unicode mapping ;12 preserve att fiphdr better
		;13-17 put inline TEXT back in.. plus allow '+' in DU
;000z	21jan99	original version
	;a/b/c/d  3jun99 small mods
	;e 17feb00 allow fiphdr on incoming file.
	;f 06mar00  maxTOlength of 400 chrs
	;g/h 10mar00 added decode and better Dispo-inline handlng
	;i 10apr00  mod for sending any file with an attachment to another DU
			NOTE need to WINNT DU and progs.DUallAtts **
	;j/k 14apr00 woops make max len of a progs.FipHdr field. (k) and filename
	;m/n 09may00 small mods and bugette in do_decode
	;o/p/q 05jun01 check_sender better; added Blacklist, strip leading NULS/SPCs
	;r 03aug01 always check for HH fields..
	;s 29oct01 WINNT filename needed better validation
	;t 24feb02 added split-every-attachment
	;u 04jun02 bugette in boundaries etc 
	;v/w/x 13jun02 if there is an SU already - use it
	;y 03oct02 bugettes - where the data does not contain a single CR or NL
	;z 23apr03 decode Headers too

(copyright) 2011 and previous years progs.FingerPost Ltd.

Notes and Comments