Allocating heap memory from within an asynchronous signal handler? Not a good idea on Mac OS.

Just this Wednesday noticed something strange in Chrome. When starting with bunch of tabs refreshing their content, Peerbelt's menu refused to appear. Recycling Chrome did not help much…

A quick glance at the Activity Monitor confirmed the Chrome process hosting Peerbelt's NPAPI was irresponsive. The sample taken revealed the following:

    +      2764 free  (in libsystem_malloc.dylib) + 277  [0x94504d1d]

    +        2764 szone_free_definite_size  (in libsystem_malloc.dylib) + 673  [0x944fada9]

    +          2764 get_tiny_previous_free_msize  (in libsystem_malloc.dylib) + 47  [0x944fef33]

    +            2764 _sigtramp  (in libsystem_platform.dylib) + 43  [0x92e81deb]

    +              2764 signalHandler(int, __siginfo*, void*)  (in PeerBelt) + 182  [0x7abbd06]  ZombieReaper.cpp:57

    +                2764 _beginthreadex(void*, unsigned int, unsigned int (*)(void*), void*, unsigned int, unsigned long*)  (in PeerBelt) + 113  [0x7abf7a1]  winapi.cpp:2033

    +                  2764 _beginthreadInternal(void*, unsigned int, void (*)(void*), unsigned int (*)(void*), void*, unsigned int, unsigned long*)  (in PeerBelt) + 688  [0x7ac1520]  winapi.cpp:1959

    +                    2764 operator new(unsigned long)  (in libc++abi.dylib) + 39  [0x9b1abf77]

    +                      2764 malloc  (in libsystem_malloc.dylib) + 52  [0x94504f44]

    +                        2764 malloc_zone_malloc  (in libsystem_malloc.dylib) + 75  [0x9450455b]

    +                          2764 ???  (in Google Chrome Framework)  load address 0x84000 + 0x818fb7  [0x89cfb7]

    +                            2764 szone_malloc  (in libsystem_malloc.dylib) + 24  [0x944f7b6a]

    +                              2764 szone_malloc_should_clear  (in libsystem_malloc.dylib) + 102  [0x94502013]

    +                                2764 _OSSpinLockLockSlow  (in libsystem_platform.dylib) + 58  [0x92e816b0]

    +                                  2764 syscall_thread_switch  (in libsystem_kernel.dylib) + 10  [0x9692c082]

Seems the OS needed a frequently called function to hook the asynchronous signal delivery for near realtime signals. But the choice of free, might not had been the best one. At very least, there may be a need to invoke _sigtramp at a different stage.

The signal handler, still within the context of mem free, tries to allocate some more memory with C++ new. Without inspecting any malloc/free code, it appears the attempt to allocate memory reverts back to the incomplete free, letting it finish the started. But the free will never complete since it is earlier in the call stack on the same thread. It all ends deadlocked.

Now, the interesting question is why it does work sometimes. My speculation is, other threads allocating/deallocating memory on the heap may be completing what the blocked thread started doing, effectively unblocking it. The deadlock manifests itself when all the other threads wait on the already locked thread. Something the call stack of the other threads does in fact suggest.

The funny thing was, I knew starting a thread just to reap a zombie process with the only benefit of a clean Activity Monitor was not such a good idea. But the implementation was so easy and in such an uninteresting from functionality perspective location, that I gave in. Mark Pincus was talking about the “Death by thousand cuts” some time back. Leaving the grand moral of the story aside, do not allocate memory on the heap from your async signal handlers! Otherwise, once in awhile there will be consequences.

A fix is now available and will be released soon, together with lookup performance improvement, adjusted Bing support on Mac and other minor UI changes. The auto updater will pick it up.

The filter bubble: Is there one? How to work around it?

Few weeks ago I read the Eli Pariser's "What is the Internet hiding from you". Found it interesting in terms of conclusions about the net effect of the information hiding. The filtering and hiding creeps in almost everywhere these days. For-profit-data analysis is here to stay. The Web is no longer the neutral, naive, democratic, and connected medium from the 90 or the early 2000s. Yeah, we have Twitter and its importance will only grow. But Twitter is all that is.

Though Eli arrives at the conclusions looking at the problem from a different than Peer Belt's angle, the end state is what really matters. Peer Belt was created to offset the negative effect of the Search Engine Optimization (SEO), democratize the search results, and give quality content publishers (small and large) a shot at having their voice heard.

But what is the SEO, really? At times, it is cheating. But deeper, focusing on the SEO result, one realizes the SEO is yet another form of information hiding technique: yeah, we know there may be better content than ours, but try to find it; we are number one in your results and " we are fine in moderation" (thank you high fructose corn syrup ad! you rock!); waste no time and stick with us!

The solution to the information hiding problem is, well, obvious once one arrives there: (anonymously?!) sharing quality, Human-curated content within a peer group. With good and timely recommendations from the peers, it would not matter much if the big sites were ranking low or hiding publicly available content from our view. It will surface.

But how does one determine what content gets shared?

It is all based on a simple idea. However, considering the idea is part of the Peer Belt's secret sauce, this particular blog may not be right place to disclose all. You can still assemble bits from the Peer Belt's product site and by running the software. The thing to remember is, there exists a way out of the filtering bubble. It involves collaboration. Whether we follow it, truly depends on us all.

Peer Belt update due in a week time

In the last couple of months Peer Belt was preparing for a major release. Support for Google Chrome has been added and is to go live with an update in about a week. The ranking has been adjusted slightly. It takes into account the impact of the words being sought and covers more data than before. The end result is quite pleasing. You should see for yourself by signing up for the beta at http://bit.ly/7AJJHD.

Internet Explorer 9: Will it reverse the trend?!

A direct answer: not so sure it will. When I first looked at the beta, I thought it had a good chance. But my perception changed since then. The reason: Microsoft is pushing the developers away (once again).

From end user perspective, it is very likely your favorite browser extension does not work well in IE9. So unless the extension developer decides to go through the pain of adjusting it, you may now have a good reason switching to Chrome, Firefox, or Flock to name few, where the same developer may have the same extension ready and waiting for you already.

As a software vendor or developer, if you happen to have software that runs into the browser as a browser helper object (a fancy name for an extension) or application automating the browser core through the ActiveX supplied, you may be in trouble. If you added mshtml.tlb to your project, so you can handle document events or manipulate the html with early bound compiled code you are definitely in trouble. You deserve some explanation at this point. Time for a little history..

When Google released the shiny new Chrome, it came with the fastest JavaScript engine called V8. Among other things V8 had tendency of shuffling object methods around, i.e. the methods/properties attached to an object, during enumeration, did not occur in the order in which they were attached to the object. Not too bad thing in a dynamic language like JavaScript. A bad thing, if transferred to objects manipulated by a statically bound at compile time vtable references.

Though I did not invest time in inspecting html object vtables and confirming with tests the order of methods in a JavaScript manipulated object is in flux in IE9, the corrupt stack asserts seen and the V8 behavior known were good enough clues. Instead of hoping to see a solution to the problem in the released IE9, decided to take action and remove any compile time dependencies on mshtml.tlb from my code. The IUnknown, IDispatch do not seem affected by the JavaScript edits (otherwise nothing would have worked). As result one still has the option of making late bound calls through IDispatch (or IDispatchEx) Invoke.

In a small test, replacing the early bound calls with invocations through IDispatch got rid of the stack corruption asserts and yielded correct results. Still the calls looked bulky with all the variant pack/unpack and QueryInterface calls. Decidde to write a small template library that would absorb all the glue code. It was the right way to go, considering the total of 20-30 early bound calls to transform. Managed to get it all covered in a day. Attaching the code to this note. Hope the code serves you well! Should you have notes about the code feel free to post them in the comments section.

--
P.S. One thing the code does not cover is a generic reader/writer for primitive types. I only needed to handle long and BSTR types. For them, instead of something truly generic, decided to write a template specialization instead. While you may create you own specialization for let's say VARIANT_BOOL a generic enough Get/Put that handles well all but the ref variant types is the path to follow.

#pragma once
#include 


#define DYNAMIC_ASSERT


// simple type assert T1 inherits from T2
template struct AssertOfType {
 static void constraints(T1* a) { T2* b = a; ( void )b;}
  AssertOfType() { void(*p)(T1*) = constraints; p = 0; }
};


class Dynamic: CComDispatchDriver {
protected:
 Dynamic(void):CComDispatchDriver(){
  ; }


public:
 Dynamic( IDispatch * pUnk ):CComDispatchDriver( pUnk ) {
  ; }

 template  static HRESULT Get( IDispatch* pSrc, LPCWSTR name, T* pRet ) {
  Dynamic d(pSrc); return d.Get( name, pRet ); }
 template  static HRESULT Put( IDispatch* pSrc, LPCWSTR name, T& val ){
  Dynamic d(pSrc); return d.Put( name, val ); }

 // working with raw IDispatch here..
 template  static HRESULT GetRef( IDispatch* pSrc, LPCWSTR name, T** pRet ){
  Dynamic d(pSrc); return d.GetRef( name, pRet ); }
 template  static HRESULT PutRef( IDispatch* pSrc, LPCWSTR name, T* val ){
  Dynamic d(pSrc); return d.PutRef( &d, name, val ); }

 template  HRESULT Get( LPCWSTR name, T* pRet ) {
  // keep the following line as is, until a generic
  // template version is not made available to the caller
  this should not compile!!!
 }

 template <> HRESULT Get( LPCWSTR name, long* pRet ) {
  CComVariant var;
  HRESULT hr = Get(name, &var);
  if ( SUCCEEDED( hr ) && ( var.vt != VT_I4 ) && ( var.vt != VT_UI4 ) ) {
   hr = var.ChangeType( VT_I4 ); }
  if ( SUCCEEDED( hr ) ) {
   *pRet = var.lVal; }
#ifdef DYNAMIC_ASSERT
  _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr;
 }


 template  HRESULT Put( LPCWSTR name, T& val ) {
  HRESULT hr = S_FALSE;
  if ( !p ) { hr = E_POINTER; }
  if ( !name ) { hr = E_INVALIDARG; }
  CComVariant v(val);
  if ( SUCCEEDED( hr ) ) { 
   hr = this->Put( name, v ); }
#ifdef DYNAMIC_ASSERT
  _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr; 
 }

 // specialization: variant handler
 template<> HRESULT Get( LPCWSTR name, VARIANT* pRet ) {
  HRESULT hr = S_FALSE;
  if ( !p ) { hr = E_POINTER; }
  if ( !name ) { hr = E_INVALIDARG; }
  if ( !pRet ) { hr = E_INVALIDARG; }
  if ( SUCCEEDED( hr ) && ( ( *pRet ).vt != VT_EMPTY ) ) { 
  hr = ::VariantClear( pRet ); }
  if ( SUCCEEDED( hr ) ) { 
   if ( wcscmp( name, L"__newEnum" ) == 0 ) {
    hr = this->GetProperty( DISPID_NEWENUM, pRet ); }
   else {
    hr = this->GetPropertyByName( name, pRet ); } }
#ifdef DYNAMIC_ASSERT
  _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr;
 }

// specialization: variant handler
 template<> HRESULT Put( LPCWSTR name, VARIANT& val ) {
  HRESULT hr = S_FALSE;
  if ( !p ) { hr = E_POINTER; }
  if ( !name ) { hr = E_INVALIDARG; }
  if ( SUCCEEDED( hr ) ) { 
   hr = this->PutPropertyByName( name, &val ); }
#ifdef DYNAMIC_ASSERT
  _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr;
 }


 // specialization: BSTR
 template <> HRESULT Get( LPCWSTR name, BSTR* pRet ) {
  HRESULT hr = S_FALSE;
  if ( !p ) { hr = E_POINTER; }
  if ( !pRet ) { hr = E_INVALIDARG; }
  if ( !name ) { hr = E_INVALIDARG; }

  CComVariant var;
  if ( SUCCEEDED( hr ) ) { hr = Get( name, &var ); }
  if ( SUCCEEDED( hr ) ) { hr = var.ChangeType( VT_BSTR ); }
  if ( SUCCEEDED( hr ) ) { 
   if ( *pRet ) { ::SysFreeString( *pRet ); *pRet = 0; }
   *pRet = var.bstrVal; var.vt = VT_EMPTY; }

#ifdef DYNAMIC_ASSERT
  _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr; 
 }

 // generic COM interface handling
 template  HRESULT GetRef( LPCWSTR name, T** pRet ) {
  AssertOfType< T, IUnknown >();
  CComVariant var;

  HRESULT hr = S_FALSE;
  if ( !p ) { hr = E_POINTER; }
  if ( !pRet ) { hr = E_INVALIDARG; }
  if ( !name ) { hr = E_INVALIDARG; }

  CComPtr pUnk;
  if ( SUCCEEDED( hr ) ) { 
   hr = GetRef( name, &pUnk ); }
  if ( SUCCEEDED( hr ) ) { 
   hr = GetQIPtr( pUnk, pRet ); }

#ifdef DYNAMIC_ASSERT
 _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr;
 }


 // specialization: IUnkown need different that the other COM interfaces handing
 template <> HRESULT GetRef( LPCWSTR name, IUnknown** pRet ) {
  CComVariant var;

  HRESULT hr = S_FALSE;
  if ( !p ) { hr = E_POINTER; }
  if ( !pRet ) { hr = E_INVALIDARG; }
  if ( !name ) { hr = E_INVALIDARG; }

  if ( SUCCEEDED( hr ) ) { hr = Get( name, &var ); }
  if ( ( SUCCEEDED( hr ) ) && ( !( ( var.vt == VT_DISPATCH ) || ( var.vt == VT_UNKNOWN ) ) ) ) {
   hr = E_NOINTERFACE; }
  if ( SUCCEEDED( hr ) ) {
   CComPtr p = ( var.vt == VT_BYREF ) ? *var.ppunkVal : var.punkVal;
   if ( *pRet ) { ( *pRet )->Release(); } 
   if ( p ) { *pRet = p.Detach(); }
   else { *pRet = 0; }
  }
#ifdef DYNAMIC_ASSERT
  _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr;
 }


 template  HRESULT PutRef( LPCWSTR name, T* val ){
  HRESULT hr = S_FALSE;
  AssertOfType< T, IUnknown >();
  if ( !p ) { hr = E_POINTER; }
  if ( !name ) { hr = E_INVALIDARG; }
  if ( SUCCEEDED( hr ) ) {
  CComVariant var( val );
  hr = this->Put( name, var ); }
#ifdef DYNAMIC_ASSERT
  _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr;
 }

 // statics pointing to the instance members
 template< class R > static HRESULT InvokeRef( IDispatch* pSrc, LPCWSTR name, R** pRet ) {
  Dynamic d(pSrc); return d.InvokeRef(name, pRet); }
 template< class T, class R > static HRESULT InvokeRef( IDispatch* pSrc, LPCWSTR name, T & arg1, R** pRet ) {
  Dynamic d(pSrc); return d.InvokeRef(name, arg1, pRet); }
 template< class T1, class T2, class R > static HRESULT InvokeRef( IDispatch* pSrc, LPCWSTR name, T1 & arg1, T2 & arg2, R** pRet ) {
  Dynamic d(pSrc); return d.InvokeRef(name, arg1, arg2, pRet); }

 template< class R > HRESULT InvokeRef( LPCWSTR name, R** pRet ) {
  HRESULT hr = S_FALSE;
  AssertOfType< R, IUnknown >();

  if ( !p ) { hr = E_POINTER; }
  if ( !pRet ) { hr = E_INVALIDARG; }
  if ( !name ) { hr = E_INVALIDARG; }

  CComVariant r;
  if ( SUCCEEDED( hr ) ) {
   hr = Invoke0(name, &r); }
  if ( SUCCEEDED( hr ) ) {
   hr = GetQIPtr< R >( r, pRet ); }

#ifdef DYNAMIC_ASSERT
  _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr;
 }

 template< class T, class R > HRESULT InvokeRef( LPCWSTR name, T & arg1, R** pRet ) {
  HRESULT hr = S_FALSE;
  AssertOfType< R, IUnknown >();

  if ( !p ) { hr = E_POINTER; }
  if ( !pRet ) { hr = E_INVALIDARG; }
  if ( !name ) { hr = E_INVALIDARG; }

  CComVariant r, varg1(arg1);
  if ( SUCCEEDED( hr ) ) {
   hr = Invoke1( name, &varg1, &r); }
  if ( SUCCEEDED( hr ) ) {
   hr = GetQIPtr< R >( r, pRet ); }

#ifdef DYNAMIC_ASSERT
  _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr;
 }

 template< class T1, class T2, class R > HRESULT InvokeRef( LPCWSTR name, T1 & arg1, T2 & arg2, R** pRet ) {
  HRESULT hr = S_FALSE;
  AssertOfType< R, IUnknown >();

  if ( !p ) { hr = E_POINTER; }
  if ( !pRet ) { hr = E_INVALIDARG; }
  if ( !name ) { hr = E_INVALIDARG; }

  CComVariant r, varg1(arg1), varg2(arg2);
  if ( SUCCEEDED( hr ) ) {
   hr = Invoke2( name, &varg1, &varg2, &r); }

  if ( SUCCEEDED( hr ) ) {
   hr = GetQIPtr< R >( r, pRet ); }

#ifdef DYNAMIC_ASSERT
  _ASSERTE( SUCCEEDED( hr ) );
#endif
  return hr;
 }

protected:
 template< class T > static HRESULT GetQIPtr( VARIANT & var, T** pRet ) {
  AssertOfType< T, IUnknown >( );
  HRESULT hr = S_FALSE; 

  CComPtr pUnk;
  if ( SUCCEEDED( hr ) ) {
   hr = GetQIPtr( var, &pUnk ); }
  if ( pRet && SUCCEEDED( hr ) ) {
   hr = GetQIPtr< T >( pUnk, pRet);  }
  return hr;
 }

 template<> static HRESULT GetQIPtr( VARIANT & var, IUnknown** pRet ) {
  HRESULT hr = S_FALSE;
  if ( !( ( var.vt == VT_DISPATCH ) || ( var.vt == VT_UNKNOWN ) ) ) {
   hr = E_NOINTERFACE; }
  if ( SUCCEEDED( hr ) ) {
   CComPtr< IUnknown > p = ( var.vt == VT_BYREF ) ? *var.ppunkVal : var.punkVal;
   if ( pRet && *pRet ) { ( *pRet )->Release(); } 
   if ( pRet  ) {
    if ( p ) { *pRet = p.Detach(); }
    else { *pRet = 0; } 
   }
  }
  return hr;
 }

 template< class T > static HRESULT GetQIPtr( IUnknown * pUnk, T** pRet ) {
  AssertOfType< T, IUnknown >();
  HRESULT hr = S_OK;
  CComQIPtr< T > p = pUnk;

  if ( p ) {
   if ( pRet ) {
    if ( *pRet ) { ( *pRet )->Release(); *pRet = 0; } 
    *pRet = p.Detach(); } }
  else if ( !pUnk ) {
   if ( pRet ) {
    if ( *pRet ) { ( *pRet )->Release(); } 
    *pRet = 0; } }
  else {
   hr = E_NOINTERFACE; }

  return hr;
 }

 template<> static HRESULT GetQIPtr< IUnknown >( IUnknown * pUnk, IUnknown ** pRet ) {
  if ( pRet && *pRet ) { ( *pRet )->Release(); *pRet = 0; } 
  if ( pRet ) { CComPtr p(pUnk); *pRet = p.Detach(); } }
 };


// this should be it

Hubble

Had to work outside of the office this morning. It did not take much hesitation and I found myself at the Liberty Science Center (LSC): free Wi Fi, good looking but otherwise not so good and too expensive cafe (while on the cafe subject, it does not open until 10am, an entire hour after LSC itself; had to buy a coffee from a vending machine).

There was an IMAX movie titled Hubble. The last time in the IMAX dome was 6-7 months ago, so decided to give it a try. The movie was just 45 min: not a big time waste, regardless of content quality.

It was great though. Guilty of not following much news apart from Techcrunch, Venturebeat, Scientific American (has anyone seen it anywhere lately?!), about two minutes in the movie, I found out that after the originally stated by NASA "nah, too risky, not gonna do it" in 2004, a service mission did actually take place last year. Not just that, during the service mission, Hubble got an infrared capability.

Why am I sharing all that, exposing my partial ignorance to news? Surprisingly, there might be a hope for us as species.

The movie mentioned about ~10,000 people were involved in the Hubble creation. Digging into my corporation experience, may be the project needed fewer people (the major project contributors may back me up here, though I am not looking for any type appraisal for this "insight"). One way or another, Hubble is one of the most considerable projects, and not just by man*hour measure our typical focus falls on. Keeping Hubble alive for few more years, until the launch of the infrared space telescope, let us collect more invaluable data and more importantly have no gaps in observation. The telescope will, with no doubt, have many implications on the future state of Physics. Another NASA project that comes to mind is Pioneer, though it might not have been a disruptive from Physics perspective until the Pioneer anomaly.

Collectively, the anomalies discovered by either projects, challenge the Physical models we've built, let us gain better understanding of the big world we live in, adjust, improve, develop new technologies, and ultimately give us a chance to survive as species in a long run.

Not knowing the development during the 2 years of "not going to do it" and not considering the motivations of the parties involved, we should all thank Sen. Mukilski for keeping one of the most significant human projects operational. Senator, thank you!

Udemy rocked the startup alley at Techcrunch Disrupt

Udemy's co-founder Gagan Biyani, after 11 straight hours talking to people in the startup alley, did the impossible: Udemy few short of just few votes joining the startup battlefield.

To understand the achievement, one has to compare how Udemy and the startup alley winner got their votes. At one point, the winning team had about five people recruiting potential voters. While they had a fine product, their focus was on the votes and the votes only. At the same time, Udemy got support thanks to just Gagan's excellent presentation and the quality of the product offered.

While the later approach did not pay off entirely yesterday, falling short of 4 votes indicates something great. And it is all well deserved, once we look at the functionality Udemy has to offer. While the site hosts prepackaged courses, real time communication tools, like chat, camera, and white board, allow for a different experience. Without them, teaching classes will be difficult. There is improved experience viewing static course content too. Video and presentation content are seen side by side. No need to switch between different applications. The content is of excellent quality, where and when you need it. With no doubt, we'll soon see and hear more about the academy of you.

Go Udemy!

Geoffrey Moore at Business of Software 2009

Great talk. Hope you enjoy it.