Troubleshooting the error message: A dynamic link library (DLL) initialization routine failed

In this post I describe the reason why an error message box was generated after I executed a DLL that I created.

The start

I was writting a piece of code on Visual Studio with ultimate goal to generate a DLL which I could use as a Proof of Concept and load it in the address space of a process. After the compile, I used rundll32.exe to execute the generated DLL and confirm it was actually working as initially designed. The DLL itself is exporting a single function. This function sends a string to the debugger in order to show that the function was indeed executed. Pretty simple, right?

In order to determine if the function was executed and the string was indeed printed, I used the DebugView utility from SysInternals. I run the application as Administrator and selected the Capture Win32 navigating to Capture -> Capture Win32 menu. What I observed was that even though the debug string was shown on Debugview, a message box was also generated:

There was a problem starting test.dll

A dynamic link library (DLL) initialization routine failed.

The DLL was working fine but this message box was annoying. And the fact I couldn’t explain why this was actually happening was making it even more annoying.

The code

The code I compiled to generated the DLL is listed in the table below.

#include <Windows.h>
#include <debugapi.h>

#define DllExport __declspec(dllexport)

DllExport void __stdcall test(void)
{
	OutputDebugStringA("test() was executed");
}

BOOL WINAPI DllMain(HINSTANCE hInstance, DWORD fdwReason, LPVOID lpvReserved)
{
	switch (fdwReason)
	{
		case DLL_PROCESS_ATTACH:
			test();
			break;
		case DLL_PROCESS_DETACH:
		case DLL_THREAD_ATTACH:
		case DLL_THREAD_DETACH:
			break;
	}

	return FALSE;
}

The reason

After some initial searching on search engines, nothing came up so I started asking around. Finally, @am0nsec gave the answer. The DllMain in the code I wrote was returning FALSE and thus was saying to the operating system that DllMain was actually failing. What I had to do was simply to change the return value to TRUE.

The lesson

When this error message is generated, it is indicative that along the way a DllMain has failed. Wanna take on the challange and identify which DLL has failed…?