| SED(1P) | POSIX Programmer's Manual | SED(1P) |
This manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ (consult the corresponding Linux manual page for details of Linux behavior), or the interface may not be implemented on Linux.
sed — stream editor
sed [-n] script [file...]
sed [-n] -e script [-e script]... [-f script_file]... [file...]
sed [-n] [-e script]... -f script_file [-f script_file]... [file...]
The sed utility is a stream editor that shall read one or more text files, make editing changes according to a script of editing commands, and write the results to standard output. The script shall be obtained from either the script operand string or a combination of the option-arguments from the -e script and -f script_file options.
The sed utility shall conform to the Base Definitions volume of POSIX.1‐2017, Section 12.2, Utility Syntax Guidelines, except that the order of presentation of the -e and -f options is significant.
The following options shall be supported:
If any -e or -f options are specified, the script of editing commands shall initially be empty. The commands specified by each -e or -f option shall be added to the script in the order specified. When each addition is made, if the previous addition (if any) was from a -e option, a <newline> shall be inserted before the new addition. The resulting script shall have the same properties as the script operand, described in the OPERANDS section.
The following operands shall be supported:
The standard input shall be used if no file operands are specified, and shall be used if a file operand is '-' and the implementation treats the '-' as meaning standard input. Otherwise, the standard input shall not be used. See the INPUT FILES section.
The input files shall be text files. The script_files named by the -f option shall consist of editing commands.
The following environment variables shall affect the execution of sed:
Default.
The input files shall be written to standard output, with the editing commands specified in the script applied. If the -n option is specified, only those input lines selected by the script shall be written to standard output.
The standard error shall be used only for diagnostic and warning messages.
The output files shall be text files whose formats are dependent on the editing commands given.
The script shall consist of editing commands of the following form:
[address[,address]]function
where function represents a single-character command verb from the list in Editing Commands in sed, followed by any applicable arguments.
The command can be preceded by <blank> characters and/or <semicolon> characters. The function can be preceded by <blank> characters. These optional characters shall have no effect.
In default operation, sed cyclically shall append a line of input, less its terminating <newline> character, into the pattern space. Reading from input shall be skipped if a <newline> was in the pattern space prior to a D command ending the previous cycle. The sed utility shall then apply in sequence all commands whose addresses select that pattern space, until a command starts the next cycle or quits. If no commands explicitly started a new cycle, then at the end of the script the pattern space shall be copied to standard output (except when -n is specified) and the pattern space shall be deleted. Whenever the pattern space is written to standard output or a named file, sed shall immediately follow it with a <newline>.
Some of the editing commands use a hold space to save all or part of the pattern space for subsequent retrieval. The pattern and hold spaces shall each be able to hold at least 8192 bytes.
An address is either a decimal number that counts input lines cumulatively across files, a '$' character that addresses the last line of input, or a context address (which consists of a BRE, as described in Regular Expressions in sed, preceded and followed by a delimiter, usually a <slash>).
An editing command with no addresses shall select every pattern space.
An editing command with one address shall select each pattern space that matches the address.
An editing command with two addresses shall select the inclusive range from the first pattern space that matches the first address through the next pattern space that matches the second. (If the second address is a number less than or equal to the line number first selected, only one line shall be selected.) Starting at the first line following the selected range, sed shall look again for the first address. Thereafter, the process shall be repeated. Omitting either or both of the address components in the following form produces undefined results:
[address[,address]]
The sed utility shall support the BREs described in the Base Definitions volume of POSIX.1‐2017, Section 9.3, Basic Regular Expressions, with the following additions:
In the following list of editing commands, the maximum number of permissible addresses for each function is indicated by [0addr], [1addr], or [2addr], representing zero, one, or two addresses.
The argument text shall consist of one or more lines. Each embedded <newline> in the text shall be preceded by a <backslash>. Other <backslash> characters in text shall be removed, and the following character shall be treated literally.
The r and w command verbs, and the w flag to the s command, take an rfile (or wfile) parameter, separated from the command verb letter or flag by one or more <blank> characters; implementations may allow zero separation as an extension.
The argument rfile or the argument wfile shall terminate the editing command. Each wfile shall be created before processing begins. Implementations shall support at least ten wfile arguments in the script; the actual number (greater than or equal to 10) that is supported by the implementation is unspecified. The use of the wfile parameter shall cause that file to be initially created, if it does not exist, or shall replace the contents of an existing file.
The b, r, s, t, w, y, and : command verbs shall accept additional arguments. The following synopses indicate which arguments shall be separated from the command verbs by a single <space>.
The a and r commands schedule text for later output. The text specified for the a command, and the contents of the file specified for the r command, shall be written to standard output just before the next attempt to fetch a line of input when executing the N or n commands, or when reaching the end of the script. If written when reaching the end of the script, and the -n option was not specified, the text shall be written after copying the pattern space to standard output. The contents of the file specified for the r command shall be as of the time the output is written, not the time the r command is applied. The text shall be output in the order in which the a and r commands were applied to the input.
Editing commands other than {...}, a, b, c, i, r, t, w, :, and # can be followed by a <semicolon>, optional <blank> characters, and another editing command. However, when an s editing command is used with the w flag, following it with another command in this manner produces undefined results.
A function can be preceded by a '!' character, in which case the function shall be applied if the addresses do not select the pattern space. Zero or more <blank> characters shall be accepted before the '!' character. It is unspecified whether <blank> characters can follow the '!' character, and conforming applications shall not follow the '!' character with <blank> characters.
If a label argument (to a b, t, or : command) contains characters outside of the portable filename character set, or if a label is longer than 8 bytes, the behavior is unspecified. The implementation shall support label arguments recognized as unique up to at least 8 bytes; the actual length (greater than or equal to 8) supported by the implementation is unspecified. It is unspecified whether exceeding the maximum supported label length causes an error or a silent truncation.
Each command in the list of commands shall be terminated by a <newline> character, or by a <semicolon> character if permitted when the command is used outside the braces. The editing commands can be preceded by <blank> characters, but shall not be followed by <blank> characters.
Long lines shall be folded, with the point of folding indicated by writing a <backslash> followed by a <newline>; the length at which folding occurs is unspecified, but should be appropriate for the output device. The end of each line shall be marked with a '$'.
If no next line of input is available, the n command verb shall branch to the end of the script and quit without starting a new cycle.
If no next line of input is available, the N command verb shall branch to the end of the script and quit without starting a new cycle or copying the pattern space to standard output.
The replacement string shall be scanned from beginning to end. An <ampersand> ('&') appearing in the replacement shall be replaced by the string matching the BRE. The special meaning of '&' in this context can be suppressed by preceding it by a <backslash>. The characters "\n", where n is a digit, shall be replaced by the text matched by the corresponding back-reference expression. If the corresponding back-reference expression does not match, then the characters "\n" shall be replaced by the empty string. The special meaning of "\n" where n is a digit in this context, can be suppressed by preceding it by a <backslash>. For each other <backslash> encountered, the following character shall lose its special meaning (if any).
A line can be split by substituting a <newline> into it. The application shall escape the <newline> in the replacement by preceding it by a <backslash>.
The meaning of an unescaped <backslash> immediately followed by any character other than '&', <backslash>, a digit, <newline>, or the delimiter character used for this command, is unspecified.
A substitution shall be considered to have been performed even if the replacement string is identical to the string that it replaces. Any <backslash> used to alter the default meaning of a subsequent character shall be discarded from the BRE or the replacement before evaluating the BRE or using the replacement.
The value of flags shall be zero or more of:
"%d\n", <current line number>
The following exit values shall be returned:
Default.
The following sections are informative.
Regular expressions match entire strings, not just individual lines, but a <newline> is matched by '\n' in a sed RE; a <newline> is not allowed by the general definition of regular expression in POSIX.1‐2008. Also note that '\n' cannot be used to match a <newline> at the end of an arbitrary input line; <newline> characters appear in the pattern space as a result of the N editing command.
When using sed to process pathnames, it is recommended that LC_ALL, or at least LC_CTYPE and LC_COLLATE, are set to POSIX or C in the environment, since pathnames can contain byte sequences that do not form valid characters in some locales, in which case the utility's behavior would be undefined. In the POSIX locale each byte is a valid single-byte character, and therefore this problem is avoided.
This sed script simulates the BSD cat -s command, squeezing excess empty lines from standard input.
sed -n '
# Write non-empty lines.
/./ {
p
d
}
# Write a single empty line, then look for more empty lines.
/^$/ p
# Get next line, discard the held <newline> (empty line),
# and look for more empty lines.
:Empty
/^$/ {
N
s/.//
b Empty
}
# Write the non-empty line before going back to search
# for the first in a set of empty lines.
p
'
The following sed command is a much simpler method of squeezing empty lines, although it is not quite the same as cat -s since it removes any initial empty lines:
sed -n '/./,/^$/p'
This volume of POSIX.1‐2017 requires implementations to support at least ten distinct wfiles, matching historical practice on many implementations. Implementations are encouraged to support more, but conforming applications should not exceed this limit.
The exit status codes specified here are different from those in System V. System V returns 2 for garbled sed commands, but returns zero with its usage message or if the input file could not be opened. The standard developers considered this to be a bug.
The manner in which the l command writes non-printable characters was changed to avoid the historical backspace-overstrike method, and other requirements to achieve unambiguous output were added. See the RATIONALE for ed for details of the format chosen, which is the same as that chosen for sed.
This volume of POSIX.1‐2017 requires implementations to provide pattern and hold spaces of at least 8192 bytes, larger than the 4000 bytes spaces used by some historical implementations, but less than the 20480 bytes limit used in an early proposal. Implementations are encouraged to allocate dynamically larger pattern and hold spaces as needed.
The requirements for acceptance of <blank> and <space> characters in command lines has been made more explicit than in early proposals to describe clearly the historical practice and to remove confusion about the phrase ``protect initial blanks [sic] and tabs from the stripping that is done on every script line'' that appears in much of the historical documentation of the sed utility description of text. (Not all implementations are known to have stripped <blank> characters from text lines, although they all have allowed leading <blank> characters preceding the address on a command line.)
The treatment of '#' comments differs from the SVID which only allows a comment as the first line of the script, but matches BSD-derived implementations. The comment character is treated as a command, and it has the same properties in terms of being accepted with leading <blank> characters; the BSD implementation has historically supported this.
Early proposals required that a script_file have at least one non-comment line. Some historical implementations have behaved in unexpected ways if this were not the case. The standard developers considered that this was incorrect behavior and that application developers should not have to avoid this feature. A correct implementation of this volume of POSIX.1‐2017 shall permit script_files that consist only of comment lines.
Early proposals indicated that if -e and -f options were intermixed, all -e options were processed before any -f options. This has been changed to process them in the order presented because it matches historical practice and is more intuitive.
The treatment of the p flag to the s command differs between System V and BSD-based systems when the default output is suppressed. In the two examples:
echo a | sed 's/a/A/p' echo a | sed -n 's/a/A/p'
this volume of POSIX.1‐2017, BSD, System V documentation, and the SVID indicate that the first example should write two lines with A, whereas the second should write one. Some System V systems write the A only once in both examples because the p flag is ignored if the -n option is not specified.
This is a case of a diametrical difference between systems that could not be reconciled through the compromise of declaring the behavior to be unspecified. The SVID/BSD/System V documentation behavior was adopted for this volume of POSIX.1‐2017 because:
The form of the substitute command that uses the n suffix was limited to the first 512 matches in an early proposal. This limit has been removed because there is no reason an editor processing lines of {LINE_MAX} length should have this restriction. The command s/a/A/2047 should be able to substitute the 2047th occurrence of a on a line.
The b, t, and : commands are documented to ignore leading white space, but no mention is made of trailing white space. Historical implementations of sed assigned different locations to the labels 'x' and "x ". This is not useful, and leads to subtle programming errors, but it is historical practice, and changing it could theoretically break working scripts. Implementors are encouraged to provide warning messages about labels that are never referenced by a b or t command, jumps to labels that do not exist, and label arguments that are subject to truncation.
Earlier versions of this standard allowed for implementations with bytes other than eight bits, but this has been modified in this version.
None.
awk, ed, grep
The Base Definitions volume of POSIX.1‐2017, Table 5-1, Escape Sequences and Associated Actions, Chapter 8, Environment Variables, Section 9.3, Basic Regular Expressions, Section 12.2, Utility Syntax Guidelines
Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1-2017, Standard for Information Technology -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 7, 2018 Edition, Copyright (C) 2018 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at http://www.opengroup.org/unix/online.html .
Any typographical or formatting errors that appear in this page are most likely to have been introduced during the conversion of the source files to man page format. To report such errors, see https://www.kernel.org/doc/man-pages/reporting_bugs.html .
| 2017 | IEEE/The Open Group |