Sunday, April 27, 2014

Zend Studio V10 Ignores Breakpoints

There are multiple causes for this problem.  The most common reason is because you are doing remote debugging and Zend Studio does not correctly identify the code base associated with the remote pages.  You can easily tell this is what is happening because when the debugger opens up the start page, the path to it will not start with your Zend Studio project name.  It will be the actual physical location or the remote server.  Further, if you set breakpoints in your project, you won’t see them in the remote project.  You can still debug in a situation like this by putting sleep(10); commands in the code, and then just manually pausing it in the debugger.  But it’s annoying, and you are better off trying to fix your project settings by manually entering the URL that matches to the project in the debug settings.

However, there is another cause of this problem which is a straight up bug in Zend Studio.  And if you use the same project for a long time, you are bound to eventually run into it.  I have seen this issue with versions of Zend Studio going back to version 7.2 (I am not saying it didn’t exist before that, but that’s the first version I can definitely say I’ve seen it in), and it still exists in version 10 (which I am currently using).  One day, Zend Studio just starts ignoring breakpoints in some specific file.  You can set breakpoints in the calling method, but if you set it in method X, it will ignore it.  If you put in the above sleep command, and then you pause it, you will see your breakpoints in the file, but Zend Studio just fails to respect them.  And the situation will gradually deteriorate until Zend Studio simply will not stop for any breakpoint anywhere.

At this point, you need to delete the project in Zend Studio (make sure you don’t delete your source files).  Now go to the directory where the project is located, and do a:

rm .buildpath
rm .project
rm –r –f .settings

And now go back to the PHP Explorer tab and recreate the project from existing source.  If you have any custom items in your PHP build path, you’ll have to redo them.  That’s unfortunate, but this is the only way I’ve found to fix this problem.  Fortunately, it’s fairly rare, and you’ll likely be able to use it for several months or even years before this starts to happen.  Because it takes so long for it to show up, and it’s seemingly random, I have no idea what causes it to happen and cannot offer any advice on how to avoid it.


But it’s definitely a bug in Zend Studio and not any kind of misconfiguration as it affects projects that have been working perfectly for months or even years in some cases.

Sunday, November 24, 2013

#define logging class for C++

The intent of this class is to lookup the identifier (the GET_USER_DATA part) by the token string (the value the text part gets converted to).

So if you have:

#define                        GET_USER_DATA      0x1FD02000

Given, “0x1FD02000”, I need to find “GET_USER_DATA”.  I tried to find some elegant want to do this, but there simply is not any possible way to do this.  The compiler replaces those identifiers at compile time.  And the information is simply not in the finished program. 

I wanted to do this to assist in debugging a multi-part networked application, for which I was not the original author.  The various application servers send data packets back and forth with the message type as the first DWORD in the packet.  I printed out the “dwMsgId” hex values from the packets, each of which corresponded to some identifier in the messages.h file, but there were so many message types that I couldn’t remember them all.  So I had to constantly keep doing searches on the Hex values to look them up and see what I was looking at.  So I tried to find some more readable way of logging these values.

One easiest solution is something like this:

switch (dwMsgId) {
   case GET_USER_DATA:
      strcpy (cText, "GET_USER_DATA");
      ...
      break;
   case NOTIFY_USER_LOG:
      strcpy (cText, "NOTIFY_USER_LOG");
      ...
      break;

But you can't swap that in and out easily between the debug and release versions, and it's just generally messy when you start talking about 80+ message types.  So instead, I decided to make a map based logging class.  And I figured I'd write a perl script to parse the message.h file to get the #defines and format them to fit the map class.  With that method, we get a call in the target application that looks something like:

#ifdef _DEBUG_XSOCKET
            wsprintf (cHex, "0X%.8X", * dwMsgId);
            cText = m_pLogXSock->getText(cHex);
            wsprintf(cLogMsg, "Message Received (0x%.8X) : %s", dwMsgId, cText );
#endif

Using this, we can set the value "_DEBUG_XSOCKET" in the preprocessor directives for the debug version of the project so the block gets automatically swapped in and out depending on the build type.

The actual class to support this follows.  Because the program I was trying to debub is an old-school C program, it uses character pointers for everything rather than std::string.  The std::map class does not support arguments of type *char by default.  So I defined a custom comparison function.  One thing to note here is that I tried to use the "unordered_map" library first.  I didn't really check it that closely, and I started getting a weird linker error about something to do with a single argument function. I then recreated this class using std::string.  But that didn't fix the problem.  So finally I broke down and read through the MSDN docs on the function.  The unordered_map library does not support customer comparison functions (i.e. <map> and <unordered_map> have different signatures).  So in the end, I went with the <map> version and swapped this back to the *char version.

The map class "find()" function either returns an iterator to the target entry, or it sends back something that is effectively an invalid pointer.  If you try and access a nonexistent entry with "whatever = it->second", it will compile fine, but it will throw an exception and crash your program when you try and use it.  So you have to compare the returned value to "tMap.end()" before you try and use it to access anything.

LogXSock.h
================
#if !defined(AFX_LOGXSOCK_H)
#define AFX_LOGXSOCK_H

#include <windows.h>
#include <stdio.h>
#include <stdlib.h>
#include "Messages.h"
#include <map>

class CLogXSock 
{
public:
       char * cGetValue(char * cXkey);    
       CLogXSock();
       virtual ~CLogXSock();

       char * cEmpty;

private:
       struct cmp_str : std::binary_function<char *, char *, bool> {
        bool operator() (char *a, char *b)  {
           return std::strcmp(a, b) < 0;
        }
    };

    std::map<char *, char *, cmp_str> tMap;
    std::map<char *, char *, cmp_str>::iterator it;
      
};

#endif // !defined(AFX_LOGXSOCK_H)

LogXSock.cpp
================
#include "LogXSock.h"

CLogXSock::CLogXSock()
{
       tMap["0X1DF00013"] = "MSGID_SERVERSTOCKMSG";
       tMap["0X1DF4D009"] = "MSGID_ITEMCREATE";
       tMap["0X1DF23500"] = "MSGID_REQUEST_REGISTERSERVER";
       tMap["0X1DF00000"] = "MSGID_SENDSERVERSHUTDOWNMSG";
       tMap["0X1DF10002"] = "MSGID_SERVERALIVE";

       cEmpty = new char[2];
       strcpy(cEmpty, "");
}

char * CLogXSock::cGetValue (char * cXkey) {
       it = tMap.find(cXkey);
       if (it != tMap.end()) {
              return(it->second);
       } else {
              return (cEmpty);
       }
}

CLogXSock::~CLogXSock()
{

}


And lastly, here is the perl script I used to find the defines and pull out the values:

extract_define_from_header.pl
========================
#!usr/bin/perl

use File::Basename;
#use Switch;
$outputFile = "map_defines.txt";
$open = "UPDATE country SET country_code = '";
$middle = "' WHERE  country_name = '";
$end = "';";
$num_args = $#ARGV + 1;

$scriptName = fileparse($0, ".pl"); #get the base name of the calling script

if ($ARGV[0] eq "-h" or $ARGV[0] eq "--h") {

  print"This script ingests a country_code file and writes it out'\n";
  print"\n";
  print"usage: $scriptName.pl input.txt\n";
  print"usage: $scriptName.pl input.txt output.txt\n";

  print"\n";
  exit;
}
if ($num_args < 1) {
   die ("Must pass the input file as the first argument.  You passed nothing.\nTry 'perl $scriptName.pl -h' for help.\nExiting...\n");
}
if ($num_args > 0) {
  $inputFile = $ARGV[0];
}
if ($num_args > 1) {
  $outputFile = $ARGV[1];
}
 
open INFILE, "$inputFile" or die "Failed to open inputFile $inputFile; Exiting...\n";
open OUTFILE, ">$outputFile" or die "Failed to open outputFile $outputFile for writing; Exiting...\n";

while ($line = <INFILE>) {
   chomp ($line);
   @split = split(/\s+/,$line);
   $define = $split[0];
   if ($define eq "#define") {
       $name = $split[1];
       $value = $split[2];
       $length = @split;
        print (OUTFILE "tMap[\"".uc($value)."\"] = \"" . uc($name) . "\";\n");
   }
}
close (INFILE);
close (OUTFILE);



Run the perl script as:

perl extract_define_from_header.pl   targetDefineFile.h outputFile.txt

Then copy the results from "outputFile.txt" and paste them into the constructor for LogXSock.cpp.

In practical application, I moved the logging into the class as well so that it reduces the footprint in the target application.  But I've found that tends not to be very portable.  So this remains the base class for this type of logging functionality.  If you use strings instead of chars, then you can just use the built in comparison functionality rather than a custom comparison function, and you can use <unordered_map> which is faster.  We don't need any type of sorting in this application, but I'm guessing, and it is a guess, that the std::map using the target application’s native *char arguments is going to be faster than a std::string based function where we have to translate back and forth, but where we can use <unordered_map> rather than <map>.  But regardless, I also have a string based version of this in my toolbox as well.




Thursday, July 18, 2013

Fix Magento catalog_product_flat Indexer Stuck on Processing

Occasionally, the magento “catalog_product_flat” indexer will hang.  Recently, this happened to a store I was working on that had several hundred thousand products with 225 catalog_product_flat_x entries. 

The fix for this problem is normally to BACKUP THE DATABASE, truncate all the “catalog_product_flat_x” tables, and then reindex the catalog product entries.

However, I am not so hot to manually truncate over 200 tables by hand.  So I used a bash script along with a configuration file.

One thing interesting to note here, I flat out could not get “mysql” on my Mac (installed with Zend server) to recognize the “mysql –defaults-extra-file=whatever” command.  It works fine on the AWS boxes, but on my Mac, it just keeps giving the error:

/usr/local/zend/mysql/bin/mysql.client: unknown variable 'defaults-extra-file=magento.cnf'

which is pretty annoying.  And yes, it was the first argument after the “mysql” command.  I created a SQL script to create a dummy local magento database with a couple “catalog_product_flat_x” tables as well as some other differently named tables to test it on.  But I ended up having to test it on an AWS dev server because of this.

Giving credit where it is due, I modified a bash script from here: https://gist.github.com/marcanuy/5977648

The modified script is:

#!/bin/bash
# Truncate database tables from console
DATABASE=magento
DEFAULTS_FILE=$DATABASE.cnf

mysql --defaults-extra-file="$DEFAULTS_FILE" -Nse 'SELECT table_name FROM information_schema.tables WHERE table_schema = "magento" AND table_name like ("catalog_product_flat_%")' $DATABASE | while read table; do mysql --defaults-extra-file="$DEFAULTS_FILE" -e "truncate table $table" $DATABASE; done

And the config file which needs to be named “magento.cnf”, looks like:
[client]
host=localhost
user=magento
password=magento

where of course you have to substitute the credentials for your mysql server in place of the test credentials above.

Then finally, go to the Magento docroot directory, and re-run the indexer:


php shell/indexer.php --reindex catalog_product_flat