Module API

[oltolm]

drmingw is now using dwarfstack, but it's using a fork because it needs an API that dwarfstack does not provide. So I have come up with a new API.

// DWST_LINE_ADDR: start address of the source line (only dwstOfModule())
//   addr:              line address
//   filename:          NULL
//   funcname:          function name
#define DWST_LINE_ADDR          -4

This is needed to calculate the displacement for SymGetLineFromAddr64.

// DWST_FUNC_ADDR: start address of the function (only dwstOfModule())
//   addr:              function address
//   filename:          NULL
//   funcname:          function name
#define DWST_FUNC_ADDR          -5

This is needed to calculate the displacement for SymGetSymFromAddr64.

// dwstOfModule(): stack information of module
//   module:            module handle from dwstModuleOpen()
//   name:              executable location
//   imageBase:         used image base address
//   addr:              stack addresses
//   count:             number of addresses
//   callbackFunc:      callback function
//   callbackContext:   user-provided pointer (context)
EXPORT int dwstOfModule(
    dwst_module *module,const char *name,uint64_t imageBase,
    uint64_t *addr,int count,
    dwstCallback *callbackFunc,void *callbackContext );

EXPORT int dwstOfModuleW(
    dwst_module *module,const wchar_t *name,uint64_t imageBase,
    uint64_t *addr,int count,
    dwstCallbackW *callbackFunc,void *callbackContext );

typedef struct dwst_module dwst_module;

// dwstModuleOpen(): prepare debug information for repeated lookups
//   dbg:               Dwarf_Debug of the executable
//                        (has to stay valid until dwstModuleClose())
//   imageBase_dbg:     preferred image base address of the executable
EXPORT dwst_module *dwstModuleOpen(
    void *dbg,uint64_t imageBase_dbg );

// dwstModuleClose(): free module created by dwstModuleOpen()
//   (dbg itself is not freed)
//   module:            module handle
EXPORT void dwstModuleClose(
    dwst_module *module );

These functions are needed for opening and closing modules. drmingw needs to be able to cache the module information. Opening and closing the module each time is too expensive.

[ssbssa]

I think there is some misunderstanding what dwstOfFileExt is supposed to do.

Looking at mgwhelp/dwarf_find.cpp, it tries to use it to find symbols by addresses, which dwarfstack just doesn't provide at all.
It only gives you source code locations.

And even dwarf_find_line seems to be wrong, because the find_line_cbW callback can be called multiple times if there are inlined functions, and that's not handled at all (and probably doesn't care for it).

In my opinion this calls for specialized functions for both of these cases, and not extending the callback with unrelated functionality.

That being said, the first 2 patches are fine in principle, but probably not very useful on its own.

[oltolm]

I think there is some misunderstanding what dwstOfFileExt is supposed to do.

Looking at mgwhelp/dwarf_find.cpp, it tries to use it to find symbols by addresses, which dwarfstack just doesn't provide at all. It only gives you source code locations.

I don't quite understand. A source location is a file name + line numer. Do you mean that it doesn't work when there are inlined functions? The last callback wins, which is the outermost function, which is correct for SymFromAddrW. There are other functions in dbghelp for inlined functions. It matches the behaviour of dbghelp.

And even dwarf_find_line seems to be wrong, because the find_line_cbW callback can be called multiple times if there are inlined functions, and that's not handled at all (and probably doesn't care for it).

The last callback wins, which is also correct for SymGetLineFromAddrW64 because it is the outermost function.

In my opinion this calls for specialized functions for both of these cases, and not extending the callback with unrelated functionality.

That being said, the first 2 patches are fine in principle, but probably not very useful on its own.

It does not care about inlined functions, that's right. The displacements are also mostly correctly calculated, with the exception of line addresses because DWARF does not have all the information, but I also don't really care about displacements.

But having said that, I am not insisting on this API. If you have a better proposal then that's even better.

[ssbssa]

I think there is some misunderstanding what dwstOfFileExt is supposed to do.
Looking at mgwhelp/dwarf_find.cpp, it tries to use it to find symbols by addresses, which dwarfstack just doesn't provide at all. It only gives you source code locations.

I don't quite understand. A source location is a file name + line numer. Do you mean that it doesn't work when there are inlined functions? The last callback wins, which is the outermost function, which is correct for SymFromAddrW. There are other functions in dbghelp for inlined functions. It matches the behaviour of dbghelp.

I mean it doesn't work for variable symbols, only functions.

And even dwarf_find_line seems to be wrong, because the find_line_cbW callback can be called multiple times if there are inlined functions, and that's not handled at all (and probably doesn't care for it).

The last callback wins, which is also correct for SymGetLineFromAddrW64 because it is the outermost function.

You're right, if you only care about the last callback, this works.
But it's also doing lots of useless stuff for this case.

In my opinion this calls for specialized functions for both of these cases, and not extending the callback with unrelated functionality.
That being said, the first 2 patches are fine in principle, but probably not very useful on its own.

It does not care about inlined functions, that's right. The displacements are also mostly correctly calculated, with the exception of line addresses because DWARF does not have all the information, but I also don't really care about displacements.

If the displacement doesn't matter, then what is the DWST_LINE_ADDR callback for?

But having said that, I am not insisting on this API. If you have a better proposal then that's even better.

That would be easier if I knew what these functions will be later used for.

[oltolm]

I think there is some misunderstanding what dwstOfFileExt is supposed to do.
Looking at mgwhelp/dwarf_find.cpp, it tries to use it to find symbols by addresses, which dwarfstack just doesn't provide at all. It only gives you source code locations.

I don't quite understand. A source location is a file name + line numer. Do you mean that it doesn't work when there are inlined functions? The last callback wins, which is the outermost function, which is correct for SymFromAddrW. There are other functions in dbghelp for inlined functions. It matches the behaviour of dbghelp.

I mean it doesn't work for variable symbols, only functions.

And even dwarf_find_line seems to be wrong, because the find_line_cbW callback can be called multiple times if there are inlined functions, and that's not handled at all (and probably doesn't care for it).

The last callback wins, which is also correct for SymGetLineFromAddrW64 because it is the outermost function.

You're right, if you only care about the last callback, this works. But it's also doing lots of useless stuff for this case.

In my opinion this calls for specialized functions for both of these cases, and not extending the callback with unrelated functionality.
That being said, the first 2 patches are fine in principle, but probably not very useful on its own.

It does not care about inlined functions, that's right. The displacements are also mostly correctly calculated, with the exception of line addresses because DWARF does not have all the information, but I also don't really care about displacements.

If the displacement doesn't matter, then what is the DWST_LINE_ADDR callback for?

It is for displacement, I just meant that DWARF only has the line address for the innermost inlined function and I am not going to try to fix it.

But having said that, I am not insisting on this API. If you have a better proposal then that's even better.

That would be easier if I knew what these functions will be later used for.

I only know of two users, drmingw and my fork of VerySleepy. Both use mgwhelp.dll in the same way. They use StackWalk64 to get the backtrace and then for each function address they get the line and symbol using SymGetLineFromAddrW64 and SymFromAddrW. Both of those functions are implemented in mgwhelp.dll for DWARF, if there is no DWARF, they delegate to dbghlp.dll.

In the future I plan to also implement SymAddrIncludeInlineTrace, SymFromInlineContext, SymGetLineFromInlineContext and SymQueryInlineTrace in mgwhelp.dll using dwarfstack.