OCS Inventory NG Forums

Hi!
I've got problems with my inventory because of this 8859 too, I think.
My computers are using Win-1251 (russian). And I get garbage in the Admin.Console instead of russian words...

P.S. Several months passed since Your post... What's about the UTF? Will the evil 8859-1 be changed to something more 'readable'?

We've found a solution... )))))
at least for the Russian language... maybe it will work with the other langugages too.

The problem consists of two parts:
1. "Garbage" instead of russian words in the console.
2. some alphabetical symbols were incorrectly shown in the "FULL INFO about the computer", replaced by an "x", i.e. there was
"Ïîëxçîâàòåëx" instead of
"Ïîëüçîâàòåëü""

The solution:

I. Garbage.

The problem was NOT at the Client side, but in the Server.
Pity, I can't attach a file here. I would post two corrected files. It would be much easier just to overwrite the bad files in your unstallation dir, than to correct the code... If somebody needs them - I can send them via e-mail. my ICQ = 493682488

Here is the manual, what parts of code you need to correct, to get all the things right: MAYBE the Administrators will use this info to correct the next release... ? )))

1.machine.php

   we have to use the utf8_decode function on the ($item->"FIELDNAME") :

    1.1 echo  "<tr>".$tdhd.$l->g(24).$tdhf.$tdhdpb.textDecode(utf8_decode($item->USERID)).$tdhfpb."</tr>";

    1.2 echo "<tr>".$tdhd.$l->g(286).$tdhf.$tdhdpb.textDecode(utf8_decode($item->OSCOMMENTS)).$tdhfpb."</tr>";

    1.3 echo  "<tr>".$tdhd.$l->g(51).$tdhf.$tdhdpb.textDecode(utf8_decode($item->WINCOMPANY)).$tdhfpb."</tr>";

    1.4 echo "<tr>".$tdhd.$l->g(348).$tdhf.$tdhdpb.textDecode(utf8_decode($item->WINOWNER)).$tdhfpb."</tr>";

    1.5 echo <td  width='85%'>".textDecode(utf8_decode($item->DESCRIPTION))."</td>";
   
    1.6 echo "$td3".utf8_decode($item->PUBLISHER)."</td>
          $td3".utf8_decode($item->NAME)."     </td>
          $td3".textDecode($item->VERSION)."  </td>  $td3".(htmlentities(textDecode($item->COMMENTS))?htmlentities(textDecode($item->COMMENTS)):"N/A")."     </td>";

2.preferences.php

     Some fileds must be recoded just ONE TIME, some - two times

    2.1. here we setup the rules for the fields, that we need to recode just once. These are the USER-defined fields and the TAG:

if($chmp == 'Tag' || $chmp == 'Fio_sotrudnika' || $chmp == 'Podrazdelenije' )
    {
    echo utf8_decode($item[$chmp])."</span></a></font></td>\n";
    }
    2.2 Recoding all the other fields:

else{                echo utf8_decode(utf8_decode($item[$chmp]))."</span></a></font></td>\n";
    }

That's all. After these changes I've finally got the RUSSIAN words at the console, instead of garbage )))


II. Replacment of some symbols in the FULL-Info.

The issue was in the Agent. And the cure must be compiled, as I can't attach a file once again )))

file ./win32_agent/include /_common\utils.h line 371:
static void cleanXml( CMarkup* pX ) {
    CString bef = pX->GetDoc();
   
    for(int i=0;i<bef.GetLength();i++) {   
        UCHAR cut = (UCHAR)bef.GetAt(i);
-        if( (cut<32 || cut>244) && cut!=10 && cut!=13 ) { 
+        if( (cut<32) && cut!=10 && cut!=13 ) {           
            bef.SetAt(i,'x');       
        }
    }
    pX->SetDoc(bef);
}

Hello!

Nice solution, may you introduce me one for agent modification on japanese signs?

Thanks in advance, Steffens.

I'm found a solution for fully UTf8 support
for win32_agent:
I got current cvs version of win32_agent

cvs -z3 -d:pserver:anonymous@ocsinventory.cvs.sourceforge.net:/cvsroot/ocsinventory co -P win32_agent

file win32_agent/include/_common/defines.h

@@ -40,7 +40,11 @@
 #define PLATFORM_NAME                  _T( "WINDOWS")
 #define USER_AGENT                             _T( "OCS-NG")
 #define HTTP_HEADERS                   _T( "Content-type: application/x-compress")
-#define XML_HEADERS                            _T( "<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>\r\n<!DOCTYPE REQUEST>\r\n")
+#ifdef _MBCS
+#define XML_HEADERS                    _T( "<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>\r\n<!DOCTYPE REQUEST>\r\n")
+#else
+#define XML_HEADERS                    _T( "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<!DOCTYPE REQUEST>\r\n")
+#endif

 #define NB_IP_MAX                              65534

file win32_agent/include/xml/Markup.cpp

@@ -737,6 +737,23 @@
                cSource = *pSource;
        }
        csText.ReleaseBuffer(nLen);
+#ifndef _MBCS  // Convert to UTF8
+       nLen=csText.GetLength();
+       // Convert to Unicode
+       int buflen=nLen+10;
+       LPWSTR buf= new WCHAR[buflen];
+       nLen = MultiByteToWideChar(CP_ACP, MB_PRECOMPOSED, csText.GetBuffer(0), nLen, buf, buflen);
+       if (nLen > 0) {
+               LPSTR utfbuf=new CHAR[nLen*2+2];
+               nLen = WideCharToMultiByte(CP_UTF8, 0, buf, nLen, utfbuf, nLen*2+2, 0, 0);
+               if (nLen > 0) {
+                       utfbuf[nLen]=0;
+                       csText = utfbuf;
+               }
+               delete [] utfbuf;
+       }
+       delete [] buf;
+#endif
        return csText;
 }

I got OCSNG_UNIX_SERVER_1.02_RC2.tar.gz (internal version 5001) from download page of this site.
file Ocsinventory.pm

@@ -116,6 +116,12 @@
     return &_end(APACHE_SERVER_ERROR);
   }

+  #Set named UTF-8
+  my $request;
+  $request = $CURRENT_CONTEXT{'DBI_HANDLE'}->prepare('SET NAMES utf8');
+  $request->execute();
+
+
   # First, we determine the http method
   # The get method will be only available for the bootstrap to manage the deploy, and maybe, sometime to give files wich will be stored in the database
   if($r->method() eq 'GET'){

for web site of ocsinventory server (ocsreports)
file preferences.php

@@ -18,7 +18,7 @@
 define("MAX_CACHED_REGISTRY", 200 );   // Max number of registry that may be returned by optimizations queries
 define("USE_CACHE", 0 );                               //Do we use cache tables ?
 define("UPDATE_CHECKSUM", 1 );                 // do we need to update software checksum when using dictionnary ?
-define("UTF8_DEGREE", 0 );                             // 0 For non utf8 database, 1 for utf8
+define("UTF8_DEGREE", 1 );                             // 0 For non utf8 database, 1 for utf8
 define("GUI_VER", "5001");                             // Version of the GUI
 define("MAC_FILE", "files/oui.txt");   // File containing MAC database
 define("TAG_LBL", "Tag");                              // Name of the tag information
@@ -340,6 +340,11 @@
                die();
        }

+       if (UTF8_DEGREE) {
+           mysql_query("SET NAMES utf8", $link);
+           mysql_query("SET NAMES utf8", $link2);
+       }
+
        $_SESSION["writeServer"] = $link2;
        $_SESSION["readServer"] = $link;
        return $link2;
@@ -761,7 +766,7 @@
                                        echo dateTimeFromMysql($item[$chmp])."</span></a></td>\n";

                                else if(!$toutAffiche)
-                                       echo $item[$chmp]."</span></a></font></td>\n";
+                                       echo textDecode($item[$chmp])."</span></a></font></td>\n";

                        }

@@ -1580,8 +1585,10 @@
   * @return Text decoded from UTF8 according to UTF8_DEGREE
   */
 function textDecode( $txt ) {
+       global $l;
        for( $i=0; $i<UTF8_DEGREE; $i++ ) {
-               $txt = utf8_decode( $txt );
+               //$txt = utf8_decode( $txt );
+               $txt = iconv("UTF-8", $l->g(0), $txt );
        }
        return $txt;
 }

very impotant moment! MySQL database must be UTF8!
for file ocsbase.sql

@@ -1,4 +1,4 @@
-CREATE DATABASE ocsweb;
+CREATE DATABASE ocsweb DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
 USE ocsweb;

 CREATE TABLE hardware (
@@ -140,7 +140,7 @@
   OS VARCHAR(255) NOT NULL,
   CONTENT LONGBLOB NOT NULL,
   PRIMARY KEY  (NAME, OS, VERSION)
-) ENGINE=MYISAM ;
+) ENGINE=INNODB ;

 CREATE TABLE inputs (
   ID INTEGER NOT NULL auto_increment,
@@ -460,8 +460,8 @@
 INSERT INTO softwares_name_cache(name) SELECT DISTINCT name FROM softwares;

 CREATE TABLE tags (
-  Tag VARCHAR(255) NOT NULL default '',
-  Login VARCHAR(255) NOT NULL default '',
+  Tag VARCHAR(100) NOT NULL default '',
+  Login VARCHAR(100) NOT NULL default '',
   PRIMARY KEY  (Tag,Login),
   KEY Tag (Tag),
   KEY Login (Login)

P.S.

Corvin1977 wrote:

We've found a solution... )))))
...
file ./win32_agent/include /_common\utils.h line 371:
static void cleanXml( CMarkup* pX ) {
    CString bef = pX->GetDoc();
   
    for(int i=0;i<bef.GetLength();i++) {   
        UCHAR cut = (UCHAR)bef.GetAt(i);
-        if( (cut<32 || cut>244) && cut!=10 && cut!=13 ) {
+        if( (cut<32) && cut!=10 && cut!=13 ) {           
            bef.SetAt(i,'x');       
        }
    }
    pX->SetDoc(bef);
}

I did first found this solution see Forums index » Developers » Bug charset cp1251 in Russian
P.P.S. Sorry, badly english...

Steffens wrote:

Can you compile the windows agent? Would be very helpful for me to test it. My Database is already UTF-8 and i can test on several clients to see what gets into the database. Thanks in advance, Steffens.

Of course! See russian OpenNetRu topic about GLPI+OCS. We'll there see WebFileRu's url to win32AgentSetup.
P.S. On OpenNetRu's forum you can write to russian

There is a new OCSNG_WINDOWS_AGENT_1.02_RC2_OTEA at the "download" section ready, were the patch for Agent-bug ('x' symbols) is included.
I checked the new release - it works fine now, at least for the russian language )))

P.S. There is a phrase "from our Russian friend Corvin1977..." So I must say, that it wasn't my service in finding that solution - it was a great work of the other people. I just placed the solution here at the forum. Just so, to be honest.

Last edited by Corvin1977 (2008-08-13 15:21:08)

I tested this agent on japanese OS and can verify now that output of agent is still not full UTF-8 which seems to be the cause for the data gets not parsed into database (already UTF-8)! as posted before!

Test output (opened in IE) of old 4032 agent as XML file gave me:

  <?xml version="1.0" encoding="ISO-8859-1" ?>
  <!DOCTYPE REQUEST (View Source for full doctype...)>
- <REQUEST>
  <DEVICEID>MEDEHHJP03-2008-08-08-19-06-06</DEVICEID>
  <QUERY>INVENTORY</QUERY>
- <CONTENT>
  <VERSIONCLIENT>4032</VERSIONCLIENT>
- <DOWNLOAD>
  <HISTORY />
  </DOWNLOAD>
- <MEMORIES>
  <CAPTION>•¨— ƒ ƒ‚ƒŠ</CAPTION>
  <DESCRIPTION>DIMM_A (No ECC)</DESCRIPTION>
  <CAPACITY>512</CAPACITY>
  <PURPOSE>System Memory</PURPOSE>

Test output (opened in IE) of newer 4050 agent as XML file gave me:

- a failure after <CAPTION> while open file in internet explorer

and the following in editor:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE REQUEST>
<REQUEST>
<DEVICEID>MEDEHHJP03-2008-08-20-12-06-00</DEVICEID>
<QUERY>INVENTORY</QUERY>
<CONTENT>
<VERSIONCLIENT>4050</VERSIONCLIENT>
<DOWNLOAD>
<HISTORY/>
</DOWNLOAD>
<MEMORIES>
<CAPTION>•¨— ƒ ƒ‚ƒŠ</CAPTION>
<DESCRIPTION>DIMM_A (No ECC)</DESCRIPTION>
<CAPACITY>512</CAPACITY>
<PURPOSE>System Memory</PURPOSE>

which is the same in XML file from older agent while opened in editor!

-> Output of this Agent is still NOT full UTF-8 conform!

(As i alredy wrote before for german characters it works now!)

May anybody out there to resolve this issue? It would really help me and maybe a lot of others too.

Thank you in advance, Steffens.

- If further testing or more information necessary, i will try to help as much as i can! -

Hi... a found an OTEA version and was hoping, this solve my problem but not. Pitty.

OTEA version still not provide UTF-8 inventory, just ANSI coded xml file with wrong charset hardcoded in it.

OCSinventroy on czech Windows XP provide file with windows-1250 encoding.

I'v tried to insert set names cp1250 to Ocsinventory.pm and propes php files, set db to UTF-8 and to cp1250 also, bud nothink work.

Have anybody client which provide UTF-8? Or, for czech and some otjher languages there should be some other simple option, because in my language special characters has some accents, abut when these accents are removed, text still is clearly readable. Is there some simple way, how to recode XML file before it is sent to server?

Or will be UTF-8 problem solved in some frinedly time? I think, on roadmap is in some stragne future...

Thanx..

Honza

This patch is only ocn server for Windows? and old version(1.01)?
It was not able to do in my environment(see below).

Ocs inventory NG Server:OCSNG_UNIX_SERVER_1.02RC3
Ocs inventory NG Agents:OCSNG_WINDOWS_AGENT_1.02RC3
Server OS:Centos 5.2
Cliant OS:Windows XP home(Japanese)
Change file:
   /usr/lib/perl5/site_perl/5.8.8/Apache/Ocsinventory.pm
----------------------------------------------------------
    ($inflated, $status) = $d->inflate($data);
    unless( $status == Z_OK or $status == Z_STREAM_END){
      if( $ENV{OCS_OPT_COMPRESS_TRY_OTHERS} ){
        &_inflate(\$raw_data, \$inflated);
      }
      else{
        undef $inflated;
      }
      if(!$inflated){
        &_log(506,'handler','Compress stage');
        return &_end(APACHE_SERVER_ERROR);
      }
    }
    $CURRENT_CONTEXT{'DATA'} = \$inflated;

#???????????????????????????????????????????????????????????????
    $data =~ s/encoding="ISO-8859-1"/encoding="windows-1251"/g;
  # change:$data=>$inflated
#???????????????????????????????????????????????????????????????

    ##########################
    # Parse the XML request
    # Retrieving xml parsing options if needed
    &_get_xml_parser_opt( \%XML_PARSER_OPT ) unless %XML_PARSER_OPT;
    unless($query = XML::Simple::XMLin( $inflated, %XML_PARSER_OPT )){
      &_log(507,'handler','Xml stage');
      return &_end(APACHE_BAD_REQUEST);
    }
----------------------------------------------------------

Is there any method of solving simply?

Hi guys
My understanding is that japanese OS is not yet supported.
Unitil now (12/12/2008) OCSagent doesn't work on a japanese winXP. Is it true?
When I run OCSinventory.exe /debug /server:xxx /nport:xxx , it doesn't generate any log on C:\Program Files\OCS Inventory Agent\
Is there a patch to fix it? Is there a way to inventory a japanese winXP PC? Even if OCSagent doesn't export inventory file, how can I manually store from scratch the inventory of the japanese PCs on the OCSserver without use the local import server feature?
Sorry, but I'm looking for a way to inventory also the japanese PCs.

I am sorry that a reply is late.
Now, my system has stopped and cannot be tried.

Is the　agent PC's inventory file generated by the local command?
   ocsinventory.exe /local /xm

If generated satisfactorily, the agent will think that he is operating normally.
The cause of fault may be because the Japanese code is used for the PC name or directory name.
(The Japanese code "デスクトップ" is contained in the log.)

OCSNG_UNIX_SERVER_1.02RC3
Change file:
   /usr/lib/perl5/site_perl/5.8.8/Apache/Ocsinventory.pm
----------------------------------------------------------
      if(!$inflated){
        &_log(506,'handler','Compress stage');
        return &_end(APACHE_SERVER_ERROR);
      }
    }
    $CURRENT_CONTEXT{'DATA'} = \$inflated;

    ###############################
     use Encode;
     $inflated =~ s/encoding="ISO-8859-1"/encoding="Shift-JIS"/g;
     Encode::from_to($inflated,'shiftjis','euc-jp');
     Encode::from_to($inflated,'euc-jp','shiftjis');
    ###############################
    ##########################
    # Parse the XML request
    # Retrieving xml parsing options if needed
    &_get_xml_parser_opt( \%XML_PARSER_OPT ) unless %XML_PARSER_OPT;
----------------------------------------------------------
I solved by this change.