Mike Samuel [Fri, 16 Sep 2011 02:05:33 +0000 (19:05 -0700)]
exp/template/html: moved error docs out of package docs onto error codes
This replaces the errStr & errLine members of context with a single err
*Error, and introduces a number of const error codes, one per
escape-time failure mode, that can be separately documented.
The changes to the error documentation moved from doc.go to error.go
are cosmetic.
Robert Griesemer [Thu, 15 Sep 2011 23:21:42 +0000 (16:21 -0700)]
godoc: use new index/suffixarray serialization code
When saving/restoring the fulltext index, the entire
respective suffixarray is now saved/restored (as opposed
to the indexed data only, and the suffixarray recreated).
This saves significant start-up time for large indexes,
at the cost of significantly larger index files.
However, it is also common to write files that
apply to multiple architectures, so a recent CL added
to go/build the ability to process comments
listing a set of conditions for building. For example:
// +build darwin freebsd openbsd/386
says that this file should be compiled only on
OS X, FreeBSD, or 32-bit x86 OpenBSD systems.
These conventions are not yet documented
(hence this long CL description).
This CL adds build comments to the multi-system
files in the core library, a step toward making it
possible to use go/build to build them.
With this change go/build can handle crypto/rand,
exec, net, path/filepath, os/user, and time.
Mike Samuel [Thu, 15 Sep 2011 15:51:55 +0000 (08:51 -0700)]
exp/template/html: pre-sanitized content
Not all content is plain text. Sometimes content comes from a trusted
source, such as another template invocation, an HTML tag whitelister,
etc.
Template authors can deal with over-escaping in two ways.
1) They can encapsulate known-safe content via
type HTML, type CSS, type URL, and friends in content.go.
2) If they know that the for a particular action never needs escaping
then they can add |noescape to the pipeline.
{{.KnownSafeContent | noescape}}
which will prevent any escaping directives from being added.
This CL defines string type aliases: HTML, CSS, JS, URI, ...
It then modifies stringify to unpack the content type.
Finally it modifies the escaping functions to use the content type and
decline to escape content that does not require it.
There are minor changes to escapeAction and helpers to treat as
equivalent explicit escaping directives such as "html" and "urlquery"
and the escaping directives defined in the contextual autoescape module
and to recognize the special "noescape" directive.
The html escaping functions are rearranged. Instead of having one
escaping function used in each {{.}} in
{{.}} : <textarea title="{{.}}">{{.}}</textarea>
a slightly different escaping function is used for each.
When {{.}} binds to a pre-sanitized string of HTML
`one < <i>two</i> & two < "3"`
we produces something like
one < <i>two</i> & two < "3" :
<textarea title="one < two & two < "3"">
one < <i>two</i> & two < "3"
</textarea>
Although escaping is not required in <textarea> normally, if the
substring </textarea> is injected, then it breaks, so we normalize
special characters in RCDATA and do the same to preserve attribute
boundaries. We also strip tags since developers never intend
typed HTML injected in an attribute to contain tags escaped, but
do occasionally confuse pre-escaped HTML with HTML from a
tag-whitelister.
Mike Samuel [Thu, 15 Sep 2011 03:40:50 +0000 (20:40 -0700)]
exp/template/html: render templates unusable when escaping fails
This moots a caveat in the proposed package documentation by
rendering useless any template that could not be escaped.
From https://golang.org/cl/4969078/
> If EscapeSet returns an error, do not Execute the set; it is not
> safe against injection.
r: [but isn't the returned set nil? i guess you don't overwrite the
r: original if there's a problem, but i think you're in your rights to
r: do so]
runtime: eliminate handle churn when churning channels on Windows
The Windows implementation of the net package churns through a couple of channels for every read/write operation. This translates into a lot of time spent in the kernel creating and deleting event objects.
Adam Langley [Wed, 14 Sep 2011 19:32:19 +0000 (15:32 -0400)]
crypto/tls: support SSLv3
It would be nice not to have to support this since all the clients
that we care about support TLSv1 by now. However, due to buggy
implementations of SSLv3 on the Internet which can't do version
negotiation correctly, browsers will sometimes switch to SSLv3. Since
there's no good way for a browser tell a network problem from a buggy
server, this downgrade can occur even if the server in question is
actually working correctly.
Rob Pike [Wed, 14 Sep 2011 00:47:59 +0000 (17:47 -0700)]
path/filepath: new signature for Walk
This one uses a closure than an interface, and is much simpler to use.
It also enables a called function to return an error and (possibly)
halt processing.
When there is an initial value, the {{if}} ends with jsCtxDivOp
since a '/' following {{.InitVal}} would be a division operator.
When there is none, the empty {{else}} branch ends with jsCtxRegexp
since a '/' would start a regular expression. A '/' could result
in a valid program if it were on a new line to allow semicolon
insertion to terminate the VarDeclaration.
There is no '/' though, so we can ignore the ambiguity.
There are cases where a missing semi can result in ambiguity that
we should report.
<script>
{{if .X}}var x = {{.X}}{{end}}
/...{{.Y}}
</script>
where ... could be /foo/.test(bar) or /divisor. Disambiguating in
this case is hard and is required to sanitize {{.Y}}.
Note, that in the case where there is a '/' in the script tail but it
is not followed by any interpolation, we already don't care. So we
are already tolerant of
<script>{{if .X}}var x = {{.X}}{{end}}/a-bunch-of-text</script>
because tJS checks for </script> before looking in /a-bunch-of-text.
This CL
- Adds a jsCtx value: jsCtxUnknown
- Changes joinContext to join contexts that only differ by jsCtx.
- Changes tJS to return an error when a '/' is seen in jsCtxUnknown.
- Adds tests for both the happy and sad cases.
Rob Pike [Mon, 12 Sep 2011 18:47:55 +0000 (11:47 -0700)]
time: make Weekday a method.
Weekday is redundant information for a Time structure.
When parsing a time with a weekday specified, it can create an
incorrect Time value.
When parsing a time without a weekday specified, people
expect the weekday to be set.
Fix all three problems by computing the weekday on demand.
This is hard to gofix, since we must change the type of the node.
Since uses are rare and existing code will be caught by the compiler,
there is no gofix module here.
Mike Samuel [Fri, 9 Sep 2011 07:07:40 +0000 (00:07 -0700)]
exp/template/html: Grammar rules for HTML comments and special tags.
Augments type context and adds grammatical rules to handle special HTML constructs:
<!-- comments -->
<script>raw text</script>
<textarea>no tags here</textarea>
This CL does not elide comment content. I recommend we do that but
have not done it in this CL.
I used a codesearch tool over a codebase in another template language.
Based on the below I think we should definitely recognize
<script>, <style>, <textarea>, and <title>
as each of these appears frequently enough that there are few
template using apps that do not use most of them.
Of the other special tags,
<xmp>, <noscript>
are used but infrequently, and
<noframe> and friend, <listing>
do not appear at all.
We could support <xmp> even though it is obsolete in HTML5
because we already have the machinery, but I suggest we do not
support noscript since it is a normal tag in some browser
configurations.
I suggest recognizing and eliding <!-- comments -->
(but not escaping text spans) as they are widely used to
embed comments in template source. Not eliding them increases
the size of content sent over the network, and risks leaking
code and project internal details.
The template language I tested elides them so there are
no instance of IE conditional compilation directives in the
codebase but that could be a source of confusion.
The codesearch does the equivalent of
$ find . -name \*.file-extension \
| perl -ne 'print "\L$1\n" while s@<([a-z][a-z0-9])@@i' \
| sort | uniq -c | sort
The 5 uses of <plaintext> seem to be in tricky code and can be ignored.
The 2 uses of <xmp> appear in the same tricky code and can be ignored.
I also ignored end tags to avoid biasing against unary
elements and threw out some nonsense names since since the
long tail is dominated by uses of < as a comparison operator
in the template languages expression language.
I have added asterisks next to abnormal elements.
26765 div
7432 span
7414 td
4233 a
3730 tr
3238 input
2102 br
1756 li
1755 img
1674 table
1388 p
1311 th
1064 option
992 b
891 label
714 script *
519 ul
446 tbody
412 button
381 form
377 h2
358 select
353 strong
318 h3
314 body
303 html
266 link
262 textarea *
261 head
258 meta
225 title *
189 h1
176 col
156 style *
151 hr
119 iframe
103 h4
101 pre
100 dt
98 thead
90 dd
83 map
80 i
69 object
66 ol
65 em
60 param
60 font
57 fieldset
51 string
51 field
51 center
44 bidi
37 kbd
35 legend
30 nobr
29 dl
28 var
26 small
21 cite
21 base
20 embed
19 colgroup
12 u
12 canvas
10 sup
10 rect
10 optgroup
10 noscript *
9 wbr
9 blockquote
8 tfoot
8 code
8 caption
8 abbr
7 msg
6 tt
6 text
6 h5
5 svg
5 plaintext *
5 article
4 shortquote
4 number
4 menu
4 ins
3 progress
3 header
3 content
3 bool
3 audio
3 attribute
3 acronym
2 xmp *
2 overwrite
2 objects
2 nobreak
2 metadata
2 description
2 datasource
2 category
2 action
Paul Lalonde [Thu, 8 Sep 2011 06:32:40 +0000 (16:32 +1000)]
Windows: net, syscall: implement SetsockoptIPMReq(), move to winsock v2.2 for multicast support.
I don't know the protocol regarding the zsyscall files which appear to
be hand-generated, so I've re-done them and added them to the change.
The linker would catch them if gc succeeded,
but too often the cycle manifests as making the
current package and the imported copy of itself
appear as different packages, which result in
type signature mismatches that confuse users.
As a crutch, add the -p flag to say 'if you see an
import of this package, give up early'. Results in
messages like (during gotest in sort):
export_test.go:7: import "sort" while compiling that package (import cycle)
export_test.go:7: import "container/heap": package depends on "sort" (import cycle)
At this point I am very confident in exp/regexp's
behavior. It should be usable as a drop-in
replacement for regexp now.
Later CLs could introduce a CompilePOSIX
to get at traditional POSIX ``extended regular expressions''
as in egrep and also an re.MatchLongest method to
change the matching mode to leftmost longest
instead of leftmost first. On the other hand, I expect
very few people to use either.
Note that this CL will break your existing code which uses
ParseCIDR.
This CL changes ParseCIDR("172.16.253.121/28") to return
the IP address "172.16.253.121", the network implied by the
network number "172.16.253.112" and mask "255.255.255.240".