dtrace discuss - Feb 2009 - how to get DTrace to use the same structure field alignment as C?

Is there a pragma to turn off padding or some other mechanism besides 
dummying up fields to make them char[] to disable padding?

Thanks!

Jeff Trawick wrote:
> Is there a pragma to turn off padding or some other mechanism besides 
> dummying up fields to make them char[] to disable padding?
Here''s a better explanation of the issue around my particular example:

I have a structure which contains some 64-bit integers.

struct some_api_rec {
int x;
int y;
char *z;
int64_t important_field;
}

In C, important_field is at offset 0x0C (no padding between z and 
important_field).
In D, important_field is at offset 0x10 (4 bytes of padding in front of 
important_field to align it for performance, apparently).

This makes it problematic to use the C structure definition in D 
scripts. The C definition can''t be changed.

If I change important_field to

char important_field[8];

the field alignment is fixed, but then important_field will be hard to 
use in D scripts.

On Wed, Feb 11, 2009 at 10:37:27AM -0500, Jeff Trawick
wrote:
> Jeff Trawick wrote:
> >Is there a pragma to turn off padding or some other mechanism besides 
> >dummying up fields to make them char[] to disable padding?
> 
> Here''s a better explanation of the issue around my particular
example:
> 
> I have a structure which contains some 64-bit integers.
> 
> struct some_api_rec {
> int x;
> int y;
> char *z;
> int64_t important_field;
> }
> 
> In C, important_field is at offset 0x0C (no padding between z and 
> important_field).
> In D, important_field is at offset 0x10 (4 bytes of padding in front of 
> important_field to align it for performance, apparently).
> 
> This makes it problematic to use the C structure definition in D 
> scripts. The C definition can''t be changed.
> 
> If I change important_field to
> 
> char important_field[8];
> 
> the field alignment is fixed, but then important_field will be hard to 
> use in D scripts.
I assume this is on an x86 box;  are you in 32-bit or 64-bit mode?

Thanks,
- jonathan

I did some experimenting, and it appears to me that C (gcc) and DTrace
behave the same way, as long as you compare the executions in the same
addressing mode.

Gcc defaulted to -m32, which pads after the pointer, because the long
long wants to align on a 64 bit boundary.  D defaults to 64 bit, so I
used the -32 option and got the same results.  The whole structure is 24
bytes, but the fields are 4, 4, 4, and 8.

Using D''s default 64 bit mode, and using the -m64 option in gcc, the
structure was still 24 bytes, but the fields were 4, 4, 8, and 8,
because the pointer is now 64 bits wide:  no padding.

Chip


> -----Original Message-----
> From: dtrace-discuss-bounces at opensolaris.org [mailto:dtrace-discuss-
> bounces at opensolaris.org] On Behalf Of Jeff Trawick
> Sent: Wednesday, February 11, 2009 9:37 AM
> To: dtrace-discuss at opensolaris.org
> Subject: Re: [dtrace-discuss] how to get DTrace to use the same
> structurefield alignment as C?
> 
> Jeff Trawick wrote:
> > Is there a pragma to turn off padding or some other mechanism
besides
> > dummying up fields to make them char[] to disable padding?
> 
> Here''s a better explanation of the issue around my particular
example:
> 
> I have a structure which contains some 64-bit integers.
> 
> struct some_api_rec {
> int x;
> int y;
> char *z;
> int64_t important_field;
> }
> 
> In C, important_field is at offset 0x0C (no padding between z and
> important_field).
> In D, important_field is at offset 0x10 (4 bytes of padding in front
of
> important_field to align it for performance, apparently).
> 
> This makes it problematic to use the C structure definition in D
> scripts. The C definition can''t be changed.
> 
> If I change important_field to
> 
> char important_field[8];
> 
> the field alignment is fixed, but then important_field will be hard to
> use in D scripts.
> 
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org

Jonathan Adams wrote:
> On Wed, Feb 11, 2009 at 10:37:27AM -0500, Jeff Trawick wrote:
>   
>> Jeff Trawick wrote:
>>     
>>> Is there a pragma to turn off padding or some other mechanism
besides
>>> dummying up fields to make them char[] to disable padding?
>>>       
>> Here''s a better explanation of the issue around my particular
example:
>>
>> I have a structure which contains some 64-bit integers.
>>
>> struct some_api_rec {
>> int x;
>> int y;
>> char *z;
>> int64_t important_field;
>> }
>>
>> In C, important_field is at offset 0x0C (no padding between z and 
>> important_field).
>> In D, important_field is at offset 0x10 (4 bytes of padding in front of
>> important_field to align it for performance, apparently).
>>
>> This makes it problematic to use the C structure definition in D 
>> scripts. The C definition can''t be changed.
>>
>> If I change important_field to
>>
>> char important_field[8];
>>
>> the field alignment is fixed, but then important_field will be hard to 
>> use in D scripts.
>>     
>
> I assume this is on an x86 box;  are you in 32-bit or 64-bit mode?
>   
yes, x86 and 32-bit

OpenSolaris 2008.11 snv_101b_rc2 X86
pfexec /usr/sbin/dtrace -32 -s ./foo.d 1234

Here''s a subset of the exact definitions, up until the point of an 
alignment discrepancy, which I''ve kludged with char[8].

typedef int64_t apr_time_t;
typedef struct request_rec request_rec;

struct request_rec {
void *pool;
void *connection;
void *server;
request_rec *next;
request_rec *prev;
request_rec *main;
char *the_request;
int assbackwards;
int proxyreq;
int header_only;
char *protocol;
int proto_num;
const char *hostname;
/* apr_time_t request_time; */ <- desired mapping
char request_time[8];

Chip Bennett wrote:
> I did some experimenting, and it appears to me that C (gcc) and DTrace
> behave the same way, as long as you compare the executions in the same
> addressing mode.
>
> Gcc defaulted to -m32, which pads after the pointer, because the long
> long wants to align on a 64 bit boundary.  D defaults to 64 bit, so I
> used the -32 option and got the same results.  The whole structure is 24
> bytes, but the fields are 4, 4, 4, and 8.
>
> Using D''s default 64 bit mode, and using the -m64 option in gcc,
the
> structure was still 24 bytes, but the fields were 4, 4, 8, and 8,
> because the pointer is now 64 bits wide:  no padding.
>   
Weird. I just installed SUNWgcc to compare, and it has the same mapping 
as SunStudioExpress.

Maybe there''s something more interesting about the full example, where 
feeding the same declarations to gcc/SunStudioExpress vs. DTrace has 
different results.

Here''s a C testcase, which with a 32-bit build shows no padding between
hostname and request_time, but 4 bytes of padding are introduced between 
those fields when the same declarations are used in D.

#include <stdio.h>
#include <stddef.h>

/* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

typedef struct apr_pool_t apr_pool_t;
typedef struct conn_rec conn_rec;
typedef struct server_rec server_rec;
typedef struct request_rec request_rec;

typedef long long apr_int64_t;
typedef apr_int64_t apr_time_t;

struct request_rec {
/** The pool associated with the request */
apr_pool_t *pool;
/** The connection to the client */
conn_rec *connection;
/** The virtual host for this request */
server_rec *server;

/** Pointer to the redirected request if this is an external redirect */
request_rec *next;
/** Pointer to the previous request if this is an internal redirect */
request_rec *prev;

/** Pointer to the main request if this is a sub-request
* (see http_request.h) */
request_rec *main;

/* Info about the request itself... we begin with stuff that only
* protocol.c should ever touch...
*/
/** First line of request */
char *the_request;
/** HTTP/0.9, "simple" request (e.g. GET /foo\n w/no headers) */
int assbackwards;
/** A proxy request (calculated during post_read_request/translate_name)
* possible values PROXYREQ_NONE, PROXYREQ_PROXY, PROXYREQ_REVERSE,
* PROXYREQ_RESPONSE
*/
int proxyreq;
/** HEAD request, as opposed to GET */
int header_only;
/** Protocol string, as given to us, or HTTP/0.9 */
char *protocol;
/** Protocol version number of protocol; 1.1 = 1001 */
int proto_num;
/** Host, as set by full URI or Host: */
const char *hostname;

/** Time when the request started */
apr_time_t request_time;

/** Status line, if set by script */
const char *status_line;
/** Status line */
int status;

/* Request method, two ways; also, protocol, etc.. Outside of protocol.c,
* look, but don''t touch.
*/

/** Request method (eg. GET, HEAD, POST, etc.) */
const char *method;
/** M_GET, M_POST, etc. */
int method_number;

};



#define SHOW_SIZE(s) \
printf("sizeof(" #s "):\t%d\n", sizeof(s))

#define SHOW_OFFSET(t, f) \
printf("offsetof(" #f "):\t%d\n", offsetof(t, f))

int main(void)
{
SHOW_SIZE(void *);
SHOW_SIZE(request_rec);

SHOW_OFFSET(request_rec, proxyreq);
SHOW_OFFSET(request_rec, proto_num);
SHOW_OFFSET(request_rec, hostname);
SHOW_OFFSET(request_rec, request_time);
SHOW_OFFSET(request_rec, status_line);
SHOW_OFFSET(request_rec, status);
SHOW_OFFSET(request_rec, method);

return 0;
}

I found the following in the man page for gcc.  The SPARC only option,
-mno-unaligned-doubles, is the default for the Sparc architecture, and
causes all long longs to be aligned on an 8 byte boundary.  However, it
also says that even if you specify -munaligned-doubles, that long longs
inside other types (i.e. structures) are still aligned on an 8 byte
boundary.  So there seems to be no way to turn that off inside a
structure.

On the other hand, the x86 only option, -mno-align-double, is the
default for the x86 architecture, and causes all long longs to _not_ be
aligned on an 8 byte boundary.  In addition, the man page doesn''t say
anything about types inside types being an exception.

IMHO, this is why your x86 gcc test is different from your D test, and
my Sparc gcc test is the same as my D test.

Chip
> -----Original Message-----
> From: dtrace-discuss-bounces at opensolaris.org [mailto:dtrace-discuss-
> bounces at opensolaris.org] On Behalf Of Jeff Trawick
> Sent: Wednesday, February 11, 2009 1:29 PM
> To: dtrace-discuss at opensolaris.org
> Subject: Re: [dtrace-discuss] how to get DTrace to use the same
> structurefield alignment as C?
> 
> Chip Bennett wrote:
> > I did some experimenting, and it appears to me that C (gcc) and
> DTrace
> > behave the same way, as long as you compare the executions in the
> same
> > addressing mode.
> >
> > Gcc defaulted to -m32, which pads after the pointer, because the
long
> > long wants to align on a 64 bit boundary.  D defaults to 64 bit, so
I
> > used the -32 option and got the same results.  The whole structure
is
> 24
> > bytes, but the fields are 4, 4, 4, and 8.
> >
> > Using D''s default 64 bit mode, and using the -m64 option in
gcc, the
> > structure was still 24 bytes, but the fields were 4, 4, 8, and 8,
> > because the pointer is now 64 bits wide:  no padding.
> >
> Weird. I just installed SUNWgcc to compare, and it has the same
mapping
> as SunStudioExpress.
> 
> Maybe there''s something more interesting about the full example,
where
> feeding the same declarations to gcc/SunStudioExpress vs. DTrace has
> different results.
> 
> Here''s a C testcase, which with a 32-bit build shows no padding
between
> hostname and request_time, but 4 bytes of padding are introduced
> between
> those fields when the same declarations are used in D.
> 
> #include <stdio.h>
> #include <stddef.h>
> 
> /* Licensed to the Apache Software Foundation (ASF) under one or more
> * contributor license agreements. See the NOTICE file distributed with
> * this work for additional information regarding copyright ownership.
> * The ASF licenses this file to You under the Apache License, Version
> 2.0
> * (the "License"); you may not use this file except in compliance
with
> * the License. You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS"
BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> 
> typedef struct apr_pool_t apr_pool_t;
> typedef struct conn_rec conn_rec;
> typedef struct server_rec server_rec;
> typedef struct request_rec request_rec;
> 
> typedef long long apr_int64_t;
> typedef apr_int64_t apr_time_t;
> 
> struct request_rec {
> /** The pool associated with the request */
> apr_pool_t *pool;
> /** The connection to the client */
> conn_rec *connection;
> /** The virtual host for this request */
> server_rec *server;
> 
> /** Pointer to the redirected request if this is an external redirect
> */
> request_rec *next;
> /** Pointer to the previous request if this is an internal redirect */
> request_rec *prev;
> 
> /** Pointer to the main request if this is a sub-request
> * (see http_request.h) */
> request_rec *main;
> 
> /* Info about the request itself... we begin with stuff that only
> * protocol.c should ever touch...
> */
> /** First line of request */
> char *the_request;
> /** HTTP/0.9, "simple" request (e.g. GET /foo\n w/no headers) */
> int assbackwards;
> /** A proxy request (calculated during
> post_read_request/translate_name)
> * possible values PROXYREQ_NONE, PROXYREQ_PROXY, PROXYREQ_REVERSE,
> * PROXYREQ_RESPONSE
> */
> int proxyreq;
> /** HEAD request, as opposed to GET */
> int header_only;
> /** Protocol string, as given to us, or HTTP/0.9 */
> char *protocol;
> /** Protocol version number of protocol; 1.1 = 1001 */
> int proto_num;
> /** Host, as set by full URI or Host: */
> const char *hostname;
> 
> /** Time when the request started */
> apr_time_t request_time;
> 
> /** Status line, if set by script */
> const char *status_line;
> /** Status line */
> int status;
> 
> /* Request method, two ways; also, protocol, etc.. Outside of
> protocol.c,
> * look, but don''t touch.
> */
> 
> /** Request method (eg. GET, HEAD, POST, etc.) */
> const char *method;
> /** M_GET, M_POST, etc. */
> int method_number;
> 
> };
> 
> 
> 
> #define SHOW_SIZE(s) \
> printf("sizeof(" #s "):\t%d\n", sizeof(s))
> 
> #define SHOW_OFFSET(t, f) \
> printf("offsetof(" #f "):\t%d\n", offsetof(t, f))
> 
> int main(void)
> {
> SHOW_SIZE(void *);
> SHOW_SIZE(request_rec);
> 
> SHOW_OFFSET(request_rec, proxyreq);
> SHOW_OFFSET(request_rec, proto_num);
> SHOW_OFFSET(request_rec, hostname);
> SHOW_OFFSET(request_rec, request_time);
> SHOW_OFFSET(request_rec, status_line);
> SHOW_OFFSET(request_rec, status);
> SHOW_OFFSET(request_rec, method);
> 
> return 0;
> }
> 
> 
> 
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org

Chip Bennett wrote:
> I found the following in the man page for gcc.  The SPARC only option,
> -mno-unaligned-doubles, is the default for the Sparc architecture, and
> causes all long longs to be aligned on an 8 byte boundary.  However, it
> also says that even if you specify -munaligned-doubles, that long longs
> inside other types (i.e. structures) are still aligned on an 8 byte
> boundary.  So there seems to be no way to turn that off inside a
> structure.
>
> On the other hand, the x86 only option, -mno-align-double, is the
> default for the x86 architecture, and causes all long longs to _not_ be
> aligned on an 8 byte boundary.  In addition, the man page doesn''t
say
> anything about types inside types being an exception.
>
> IMHO, this is why your x86 gcc test is different from your D test, and
> my Sparc gcc test is the same as my D test.
>   
Right... Now I know how to make my structure offsets the same in C as 
they are in D:

gcc -malign-double c_offsets.c

(not that this solves my problem, but it is helpful to recall the 
differences in alignment requirements, and thus default compiler 
behavior, between SPARC and x86; perhaps DTrace doesn''t have the same 
distinction?)

Thanks!

Jeff Trawick wrote:
> Chip Bennett wrote:
>> I found the following in the man page for gcc. The SPARC only option,
>> -mno-unaligned-doubles, is the default for the Sparc architecture, and
>> causes all long longs to be aligned on an 8 byte boundary. However, it
>> also says that even if you specify -munaligned-doubles, that long longs
>> inside other types (i.e. structures) are still aligned on an 8 byte
>> boundary. So there seems to be no way to turn that off inside a
>> structure.
>>
>> On the other hand, the x86 only option, -mno-align-double, is the
>> default for the x86 architecture, and causes all long longs to _not_ be
>> aligned on an 8 byte boundary. In addition, the man page
doesn''t say
>> anything about types inside types being an exception.
>>
>> IMHO, this is why your x86 gcc test is different from your D test, and
>> my Sparc gcc test is the same as my D test.
> Right... Now I know how to make my structure offsets the same in C as 
> they are in D:
>
> gcc -malign-double c_offsets.c
>
> (not that this solves my problem, but it is helpful to recall the 
> differences in alignment requirements, and thus default compiler 
> behavior, between SPARC and x86; perhaps DTrace doesn''t have the
same
> distinction?)
More formally, 4-byte alignment for 64-bit integers is part of the ILP32 
ABI.

There is a nice table at

http://developers.sun.com/solaris/articles/about_amd64_abi.html

showing size and alignment of various data types for 32-bit and 
64-bit/x86 and SPARC.