1 APACHE COMMONS: serf -*-indented-text-*-
9 4. Bucket Read Functions
14 -----------------------------------------------------------------------------
18 This document details various design choices for the serf library. It
19 is intended to be a guide for serf developers. Of course, these design
20 principles, choices made, etc are a good source of information for
21 users of the serf library, too.
24 -----------------------------------------------------------------------------
28 The serf library should contain no mutable globals, making it is safe
29 to use in a multi-threaded environment.
31 Each "object" within the system does not need to be used from multiple
32 threads at a time. Thus, they require no internal mutexes, and can
33 disable mutexes within APR objects where applicable (e.g. pools that
36 The objects should not have any thread affinity (i.e. don't use
37 thread-local storage). This enables an application to use external
38 mutexes to guard entry to the serf objects, which then allows the
39 objects to be used from multiple threads.
42 -----------------------------------------------------------------------------
46 For general information on the proper use of pools, please see:
48 http://cvs.apache.org/viewcvs/*checkout*/apr/docs/pool-design.html
50 Within serf itself, the buckets introduce a significant issue related
51 to pools. Since it is very possible to end up creating *many* buckets
52 within a transaction, and that creation could be proportional to an
53 incoming or outgoing data stream, a lot of care must be take to avoid
54 tying bucket allocations to pools. If a bucket allocated any internal
55 memory against a pool, and if that bucket is created an unbounded
56 number of times, then the pool memory could be exhausted.
58 Thus, buckets are allocated using a custom allocator which allows the
59 memory to be freed when that bucket is no longer needed. This
60 contrasts with pools where the "free" operation occurs over a large
61 set of objects, which is problematic if some are still in use.
63 ### need more explanation of strategy/solution ...
66 -----------------------------------------------------------------------------
68 4. BUCKET READ FUNCTIONS
70 The bucket reading and peek functions must not block. Each read
71 function should return (up to) the specified amount of data. If
72 SERF_READ_ALL_AVAIL is passed, then the function should provide
73 whatever is immediately available, without blocking.
75 The peek function does not take a requested length because it is
76 non-destructive. It is not possible to "read past" any barrier with a
77 peek function. Thus, peek should operate like SERF_READ_ALL_AVAIL.
79 The return values from the read functions should follow this general
82 APR_SUCCESS Some data was returned, and the caller can
83 immediately call the read function again to read
86 NOTE: when bucket behavior tracking is enabled,
87 then you must read more data from this bucket
88 before returning to the serf context loop. If a
89 bucket is not completely drained first, then it is
90 possible to deadlock (the server might not read
91 anything until you read everything it has already
94 APR_EAGAIN Some data was returned, but no more is available
95 for now. The caller must "wait for a bit" or wait
96 for some event before attempting to read again
97 (basically, this simply means re-run the serf
98 context loop). Though it shouldn't be done, reading
99 again will, in all likelihood, return zero length
100 data and APR_EAGAIN again.
102 NOTE: when bucket behavior tracking is enabled,
103 then it is illegal to immediately read a bucket
104 again after it has returned APR_EAGAIN. You must
105 run the serf context loop again to (potentially)
106 fetch more data for the bucket.
108 APR_EOF Some data was returned, and this bucket has no more
109 data available and should not be read again. If you
110 happen to read it again, then it will return zero
111 length data and APR_EOF.
113 NOTE: when bucket behavior tracking is enabled,
114 then it is illegal to read this bucket ever again.
116 other An error has occurred. No data was returned. The
117 returned length is undefined.
119 In the above paragraphs, when it says "some data was returned", note
120 that this could be data of length zero.
122 If a length of zero is returned, then the caller should not attempt to
123 dereference the data pointer. It may be invalid. Note that there is no
124 reason to dereference that pointer, since it doesn't point to any
127 Any data returned by the bucket should live as long as the bucket, or
128 until the next read or peek occurs.
130 The read_bucket function falls into a very different pattern. See its
131 doc string for more information.
134 -----------------------------------------------------------------------------
138 The serf project uses the APR versioning guidelines described here:
140 http://apr.apache.org/versioning.html
143 -----------------------------------------------------------------------------
147 ### flesh out. basically: if you hold a bucket pointer, then you own
148 ### it. passing a bucket into another transfers ownership. use barrier
149 ### buckets to limit destruction of a tree of buckets.
152 -----------------------------------------------------------------------------