FilterPlugin

On this page:
Powered by
WikiRing Consultants

Description

This plugin allows to substitute and extract information from content by using regular expressions. There are three different types of new functions:
  1. FORMATLIST: maniplulate a list of items; it is highly configurable to define what constitutes a list and how to extract items from it
  2. SUBST, STARTSUBST/STOPSUBST: substiture a pattern in a chunk of text
  3. EXTRACT, STARTEXTRACT/STOPEXTRACT: extract a pattern from a text
While the START-STOP versions of SUBST and EXTRACT work on inline text, the normal versions process a source topic before including it into the current one.

Syntax Rules

SUBST

Syntax: %SUBST{topic="..." ...}%

insert a topic by processing its content.

STARTSUBST, STOPSUBST

Syntax: 
%STARTSUBST{...}% 
... 
%STOPSUBST%

substitute text given inline. see SUBST.

EXTRACT

Syntax: %EXTRACT{topic="..."  ...}%

extract text from a topic. see SUBST.

STARTEXTRACT, STOPEXTRACT

Syntax: 
%STARTEXTRACT{...}% 
... 
%STOPEXTRACT%

extract content given inline. see SUBST.

FORMATLIST

Syntax: %FORMATLIST{"<list>" ...}%

formats a list of items. The <list> argument is separated into items by using a split expression; each item is matched agains a pattern and then formatted using a format string while being separated by a separator string; the result is prepended with a header and appended with a footer in case the list is not empty.

The pattern string shall group matching substrings in the list item to which you can refer to by using $1, $2, ... in the format string. Any format string (format, header, footer) may contain variables $percnt$, $nop, $dollar and $n. The variable $index referse to the position number within the list being formatted; using $count in the footer or header argument refers to the total number of list elements; $marker is set if the selection regular expression matches the current item. The $map(key) macro returns the value for "key" as specified in the map argument.

MAKEINDEX

Syntax: %MAKEINDEX{"<list>" ...}%

formats a list into a multi-column index like in MediaWiki's category topcis. MAKEINDEX insert capitals as headlines to groups of sorted items. It will try to balance all columns equally, and keep track of breaks to prevent "schusterkinder", that is avoid isolated headlines at the bottom of a column.

parameters:

Like in FORMATLIST the format parameter can make use of $1, $2, ... variables to match the groupings defined in the pattern argument (like in pattern="(.*);(.*);(.*)") . The first matched grouping $1 will be used as the $item to sort the list.

In addition header and footer might contain the $anchors variable which will expand to a navigation to jump to the groups within the index.

Examples

EXTRACT Example 1: convert table into text

One of the uses of this plugin is to extract data from tables, which is useful for creating "database-like" wiki applications where data is stored in foswiki tables. While it is certainly possible to do that without this plugin the plugin makes these requests easier to create and maintain. Note, however, that best practice is to store database-like information using DataForms, so that you don't need to parse the format of the data to extract its records repeatedly.

The table:

Pos Description Hours
1 onsite troubleshooting 3
2 normalizing data to new format 10
3 testing server performance 5

You type: 

%EXTRACT{topic="%TOPIC%" expand="off" 
  pattern="^\|\s\s(.*?)\s*\|\s*(.*?)\s*\|\s*(.*?)\s*\|" 
  format="   * it took $3 hours $2$n"
  skip="1"
}%

Expected result (simulated):

Actual result (this site):

EXTRACT Example 2: convert text into table

Use CSS tags to format text comments as a tabular data (e.g., to allow sorting).

The comments:

This is the first comment.
-- Michael Daum on 22 Aug 2005

This is the second comment.
-- Michael Daum on 22 Aug 2005

You type: 

%EXTRACT{
   topic="%TOPIC%" expand="off"
   pattern=".div class=\"text\">.*?[\r\n]+(.*?)[\r\n]+(?:.*?[\r\n]+)+?-- (.*?) on (.*?)[\r\n]+"
   format="| $3 | $2 | $1 ... |$n" header="|*Date*|*Author*|*Headline*|$n"
}%

Expected result (simulated):

Date Author Headline
22 Aug 2005 Michael Daum This is the first comment. ...
22 Aug 2005 Michael Daum This is the second comment. ...

Actual result (this site):

Date Author Headline
22 Aug 2005 Michael Daum This is the first comment. ...
22 Aug 2005 Michael Daum This is the second comment. ...

MAKEINDEX example 1: creating an index from a chunk of text

compare with Philosophy articles needing attention

A B C D E F H I K L M N O P R S T U V W m p t

A

B

C

D

E

F

H

I

I

K

L

M

N

O

P

P

R

S

T

U

V

W

m

p

t

MAKEINDEX example 2: creating an index for a search result

A

A

Plugin Installation Instructions

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server.

Open configure, and open the "Extensions" section. Use "Find More Extensions" to get a list of available extensions. Select "Install".

If you have any problems, or if the extension isn't available in configure, then you can still install manually from the command-line. See http://foswiki.org/Support/ManuallyInstallingExtensions for more help.

Plugin Info

Plugin Author: Michael Daum
Copyright ©: 2005-2010, Michael Daum http://michaeldaumconsulting.com
License: GPL (GNU General Public License)
Release: 2.03
Version: 8284 (2010-07-23)
Change History:  
23 Jul 2010: fixes wrapper for non-official api call to getAnchorName on foswiki-1.1
07 Jun 2010: fixed expanding standard escapes ($n, $percent, ...); improved examples in docu
12 Feb 2010: ease tokenize; forward compatibility for newer foswikis
17 Nov 2009: added tokenize pattern for FORMATLIST; fixed potential deep recursion in SUBST/EXTRACT
14 Sep 2009: added include counterpart to already existing exclude params; fixed SUBST not to forget about the non-matching tail of a char sequence
17 Apr 2009: converted to foswiki, added numerical sorting to MAKETEXT
08 Oct 2008: added $anchors to MAKEINDEX (by Dirk Zimoch); added nocase option to FORMATLIST (by Dirk Zimoch); fixed null/empty string match in FORMATLIST
20 Aug 2008: added selection and marker to FORMATLIST, similar in use as VarWEBLIST
03 Jul 2008: sorting a list before, not after, formatting it in FORMATLIST
08 May 2008: added 'text' parameter to SUBST and EXTRACT; fixed SUBST as it was pretty useless before
07 Dec 2007: added MAKEINDEX, added lazy compilation
14 Sep 2007: added sorting for EXTRACT and SUBST
02 May 2007: using registerTagHandler() as far as possible; enhanced parameters to EXCTRACT and SUBST
05 Feb 2007: fixed escapes in format strings; added better default value for max number of hits to prevent deep recursions on bad regexpressions
22 Jan 2007: fixed SUBST, added skip parameter to FORMATLIST
18 Dec 2006: using registerTagHandler for FORMATLIST
13 Oct 2006: fixed limit parameter in FORMATLIST
31 Aug 2006: added NO_PREFS_IN_TOPIC
15 Aug 2006: added use strict; and fixed revealed errors
14 Feb 2006: moved in FORMATLIST from the Foswiki:Extensions/NatSkinPlugin; added escape variables to format strings
06 Dec 2005: fixed SUBST not to cut off the rest of the text
09 Nov 2005: fixed deep recursion using expand="on"
22 Aug 2005: Initial version; added expand toggle
Dependency: $Foswiki::Plugins::VERSION 1.024
CPAN Dependencies: none
Other Dependencies: none
Perl Version: 5.005
Plugin Home: Foswiki:Extensions/FilterPlugin
Support: Foswiki:Support/FilterPlugin
spacer