libsec: avoid undefined C
gcc compiles `p + length < p' into 'length < 0' since pointer overflow is undefined behavior in C. This breaks the check against a large `length'.
Use `length > pend - p' instead.
There's no need to check `length < 0' since `length' is from length_decode() and should be non-negative.
===
Try the simplified code.
void bar(void);
void foo(unsigned char *p, int length)
{
if (p + length < p)
bar();
}
$ gcc -S -o - t.c -O2
...
foo:
.LFB0:
.cfi_startproc
testl %esi, %esi
js .L4
rep
ret
.L4:
jmp bar
.cfi_endproc
Clearly `p' is not used at all.
R=rsc
CC=plan9port.codebot
https://codereview.appspot.com/7231069
diff --git a/src/libsec/port/x509.c b/src/libsec/port/x509.c
index 60b3d07..2bbf853 100644
--- a/src/libsec/port/x509.c
+++ b/src/libsec/port/x509.c
@@ -2077,8 +2077,7 @@
if(tag_decode(&p, pend, &tag, &isconstr) != ASN_OK ||
tag.class != Universal || tag.num != SEQUENCE ||
length_decode(&p, pend, &length) != ASN_OK ||
- p+length > pend ||
- p+length < p)
+ length > pend - p)
return;
info = p;
if(ber_decode(&p, pend, &elem) != ASN_OK || elem.tag.num != SEQUENCE)
