Patches to make an H3 boot

[sirwio]

Hi,

I have been struggling the last week to try to patch the vomp development environment to create a bootable dongle. Carefully examining the steps used in the mvpmc project I did the following.
Kernel patches - These are also used by the mvpmc project and can be found in their git repository.
1. hcwmp_header.patch
1. squashfs_2.2r2.patch
2. zlib.patch

Instead of extracting the kernel modules from a hauppauge dongle I use the one existant in the mvpmc git repository

One also need to change the dongle_build.sh script to copy the kernel modules mentioned above and create a squashfs filesystem. The mksquashfs tool to use is the one used by the mvpmc project. Its squashfs2.2-r2 thats used but its patched to take -if argument that allows one to create device inodes on the filesystem.

I'm attaching the a tar'ed set of files that shall be untared in the root of the vomp project. Run the kernel_patches.sh script to apply the patches. Note that the mksquashfs tool is not included!

I need some feedback though since the build dongle is not 100% functional - I cannot watch live tv wich is sort of essential. The strange thing is that if I use the mvpmc build suit and substitute the mvmpc client sources for vomp client sources I get a dongle with wich live tv can be seen!

The message I get, with the patched vomp buildsystem, is an dialog saying Connection Lost when I try to view a live tv channel.

Please give feedback. If someone knows a way or has ideas on howto extract the kernel modules from the hauppauge dongle with squashfs I'm willing to give it a shot.

[MartenR]

Hi,
I have only one idea.
Are you using a current cvs snapshoot? (get from cvs after saterday)
If yes, then it is clear, because Chris is currently working on the Protocoll, especielly on things connected with playback, so it might be that a cvs is currently not working.

Normally a connection lost usually indicate an error in the protokoll or in the TCP connection code. (I have seen in the git repos also a patch of the network chip (smc91111.patch), maybe you have to apply it also.

Marten

[sirwio]

Thanks for the reply,

I'm using vompserver and client sources check out from 2006-08-01 (cvs update -D 2006-08-01) which essentially should be the same as the 0.2.4 release. (I suggest that the cvs sources are tagged in forthcomming releases :-)). Note that the sources work if compiled with the mvpmc development suite.

I also doubt that the smc91111.patch solves the connection lost problem as that patch is only applied on the 2.4.31 kernel in the mvpmc project and I compiled using the 2.4.17 kernel.

Would it be of any help attaching the trace I get by running ./vompclient -d or the vompserver log? I tried this morning changing the tcp receive buffer to no avail but causing crash when the buffer was set to 1024. All other buffers caused the connection lost dialog.

One thing thats bothered me while working on this was the devtable file passed as argument to the mksquashfs tool. What devices shall really be in /dev ? Could someone perhaps post an 'ls -l /dev'

-Magnus S.

[sirwio]

Hi again,

Carefully reading the mvpmc dev mailing list I just realized that the only patch needed to the kernel is the hcwmp_header.patch This is a good thing...
So all parts in the patches that relate to squashfs can actually be ignored. One must however still use the pre extracted proprietary kernel modules since I still have no clue on howto extract them from the new hauppague dongles.

Chris/Mark: Any chance of getting the hcwmp_header.patch into the makedevenv-2 script If I understand things correctly the patch doesn't cause any problems for older hardware. Needs to be tested by someone though...

Chris/Mark: Do you want me to create a new patch wich only contains the smallest amount of changes to build a Hx bootable dongle.

The squashfs support is perhaps something that might be of interest later...

My previos request for an 'ls -l /dev' can be ignored. When I reverted the dongle_build.sh script back to create an ext2 filesystem I noticed the use of the spec file. The spec file is basically the same as the devtable file used as input to the patched mksquashfs command.

I'm still however having problem getting any livetv to work even with only the hcwmp_header.patch applied to the kernel.

- Magnus S.

[MarkC]

Thanks for all the work you're doing on this: it's much appreciated!

Yes please, if you could create a new patch with just the required changes, that'd be good. Then we'll test it on the older hardware.

Regarding the dongle build method, squashfs or whatever, I was talking to Chris a couple of weeks ago and he mentioned that sometime he'd like to investigate the various options, find out what they actually are and how they work, and decide what's best and easiest to use. In the time being, if we can get it working with the existing script, that'd be great!

Your live TV problem: yes, please attach the client and server logs. Does everything else work apart from live TV? You haven't seen any problems at all with playing back recordings?

[sirwio]

The only required change is the hcwmp_header.patch
The patch can be found attached to my previous posting or from the mvpmc git repository. Apply the patch rebuiild the kernel and test to see if it works on your hardware. I would be suprised if it caused any trouble since its allways applied on the kernels build for the mvpmc project.

Instead of using the newer hauppauge dongles I downloaded an old one and have no problems with the rest of the build. Would be nice though if someone could actually come out with a way to extract the kernel modules from a newer kernel!

I have now gotten it to show livetv using a checkout from 2006-10-01 where I have made this small change:

myth@sam ~ $ cd vomp
myth@sam ~/vomp $ cd client
myth@sam ~/vomp/client $ cvs diff
? vompclient
cvs diff: Diffing .
Index: main.cc
===================================================================
RCS file: /cvsroot/vomp/client/main.cc,v
retrieving revision 1.28
diff -r1.28 main.cc
193c193
<   success = mtd->init("/dev/mtd1");
---
>   success = 1; // mtd->init("/dev/mtd1");
215c215
<   UCHAR videoFormat = (UCHAR)mtd->getPALorNTSC();
---
>   UCHAR videoFormat = Video::PAL; // (UCHAR)mtd->getPALorNTSC();
404c404
<     mtd->shutdown();
---
>     // mtd->shutdown();
Index: vdr.cc
===================================================================
RCS file: /cvsroot/vomp/client/vdr.cc,v
retrieving revision 1.39
diff -r1.39 vdr.cc
292c292
<   int b = clock_settime(CLOCK_REALTIME, &currentTime);
---
>   int b =  0; // clock_settime(CLOCK_REALTIME, &currentTime);
cvs diff: Diffing fonts
cvs diff: Diffing other
myth@sam ~/vomp/client $

Watching recordings does not work. When the recordings menu is entered the vompclient segfaults.

I haven't had time to investigate the result of the clock_settime call yet. I will investigate this over the weekend.

- Magnus S.

[Fourty2]

Instead of using the newer hauppauge dongles I downloaded an old one and have no problems with the rest of the build. Would be nice though if someone could actually come out with a way to extract the kernel modules from a newer kernel!

So, here we go:
1. You need to build a new kernel (maybe not, if unsquashfs supports squashfs 2.x by now)
    - Get SquashFS-Kernel-Patch from
      http://prdownloads.sourceforge.net/squashfs/squashfs3.1-r2.tar.gz?download
    - Patch your kernel, build, install

2. There is a tool, called "find-squashfs", that can extract the squashfs-ram-disk. As far as I know,
    the only source is the german http://www.ip-phone-forum.de, so I append source
    and makefile.

3. Build find-squashfs and run it on a new dongle.bin to extract the squashfs ram-disk.

4. Mount this extracted squashfs-ramdisk with loop-device.

That was easy, wasn't it... ;-)

Fourty2

(find-squashfs.c)

Code Select Expand


#include <stdio.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <endian.h>
#include <stdlib.h>

#include <errno.h>

#ifndef BYTE_ORDER
#error "byte order not defined"
#endif

static void usage(name)
char name[];
{
        fprintf(stderr,"Usage: %s kernel.image\n",name);
        exit(1);
}

int main(argc,argv)
int argc;
char **argv;
{
        FILE *fd;
        long size, count;
        uint32_t *start,*point;

        /* start signature of squashfs */
#if BYTE_ORDER != LITTLE_ENDIAN
        uint32_t search = 0x73717368;
#else
        uint32_t search = 0x68737173;
#endif

        if ( argc != 2 )
                usage(argv[0]);
        fd = fopen(argv[1],"r");
        if ( fd == NULL )
        {
                fprintf(stderr,"Unable to open <%s> errno: %d\n",argv[1],errno);
                usage(argv[0]);
        }
        /* get size */
        size=fseek(fd, 0L, SEEK_END);
        size=ftell(fd);
        fprintf(stderr,"The size of <%s> is %ld\n",argv[1],size);
        fseek(fd, 0L, SEEK_SET);

        start=(uint32_t *)malloc(size);

        fread(start,size,1,fd);
        fclose(fd);

        for ( count=0,point=start; count < size; count+=sizeof(uint32_t),++point)
        {
                if ( *point == search )
                {
                        fprintf(stderr,"Squashfs signature found at %ld\n",count);
                        break;
                }
        }
        if ( count < size )
        {
                fd=fopen("kernel.raw","w");
                if ( fd == NULL )
                {
                        fprintf(stderr,"Create kernel.raw: errno %d\n",errno);
                        exit(99);
                }
                fwrite(start,1,count,fd);
                fclose(fd);
                fprintf(stderr,"kernel.raw created\n");
                fd=fopen("kernelsquashfs.raw","w");
                if ( fd == NULL )
                {
                        fprintf(stderr,"Create kernelsquashfs.raw: errno %d\n",errno);
                        exit(99);
                }
                fwrite(point,1,size-count,fd);
                fclose(fd);
                fprintf(stderr,"kernelsquashfs.raw created\n");
        }
        else
        {
                fprintf(stderr,"Strange, no squashfs signature found...\n");
                exit(99);
        }

        free(start);
        exit(0);
}


(Makefile)

Code Select Expand


CC = gcc
LD = ld
CFLAGS = -W -Wall -O2
PROGS = find-squashfs
OBJS += find-squashfs.o

all: $(OBJS)
        $(CC) -o find-squashfs $(OBJS)

clean:
        rm -f $(PROGS) $(OBJS)


.PHONY: all clean


[sirwio]

Thanks Forty2 for the outline of grabbing the kernels from the newer dongles. I'll try the tip. I had it all figured out except for the find-squashfs part...

Regarding the livetv problem I'm getting more and more confused. For some reason I had commented out the call to clock_settime when I build using the mvpmc devenv a couple of weeks ago. Now when I dig deeper into the problem I found that its enough for me to log into the mvp and set the date/time to be anything before the time the show to watch started. Lets say that the current program on channel 4 started at 2006-10-08 18.20 and then setting the date/time with 'date 100718152006' from the console before entering channel 4 brings up livetv.

Its gotta be something with the epg code or epg data! If I disable the code in VVideoLive::doBanner I can watch livetv without problems.

Right now I'm uncertain wether the problems I see are due to my H3 hardware, my vdr setup or with the vompclient/vompserver codebase!

Issues:
* The above mentioned problem with VVideoLive::doBanner code that causes connection lost on my client.

* I cannot choose the first channel in my vdr list. I can watch it without only problems on the vdr machine but when I choose it on the vompclient I get a "No such channel" dialog.

* The channel numbers are not as setup on the vdr machine. I have six channels setup with channel numbers from 1 to 6. In the vompclient channel menu these channels are listed with numbers 0 2 3 4 5 6. Its when I choose channel 0 I get the No such channel.

* When I enter the recordings menu the vompclient segfaults.

The issues above are for a cvs update from  2006-10-07

The corresponding functionallity when performed on my vdr box works fine!

Hints are welcome. What problem shall I attack first?

- Magnus S.

[MartenR]

Well, the recordings menu problem can be connected with the issue, that Chris is working on the protocoll for the recordings information (in cvs is writen that this is not finished) and you have at least to update also your server code, but it can be that the current cvs is simply not working for recordings.
Maybe you can checkout the cvs from 30.9.06 (I don't know if this is possible), this should be compatible to your 0.2.4 except for the fast forward stuff, so it would be the best to check out the server side also from 30.9. .

Regarding the live playback, I have no clue, I think if server and cleint are using the same protocoll it should work. But connection lost is typical for protocoll incompatibility, maybe server and cleint do not match or you are using compiler with which code is generate, that behaves differently from the code generate with the compiler which Chris uses. (This happend several times while writing the windows client)

Marten

[Fourty2]

Well, the recordings menu problem can be connected with the issue, that Chris is working on the protocoll for the recordings information (in cvs is writen that this is not finished) and you have at least to update also your server code, but

This definitly is the reason. With my "old" vompclient checkout (End of August) everything works fine with the patches
above (except the time-stuff patche(s)).

However, excessive jumping in record-playback-mode sometimes causes crashes...

Fourty2

[sirwio]

Hi,

My problems were related to faults in the vompserver code when running on 64 bit hardware. Please see my other thread for quick fix. Livetv, recordings, epg etc works fine for me now on a 'cvs update -D 2006-09-30' of vompclient and vompserver.

Thanks everybody who gave input...

- Magnus S.

[meando]

Hi Fourty2 and Sirwio,

any chance that you could make your dongle.bin available for the people that are less gifted in the use of compilers?
I also have a H3 that doesn't boot the current VOMP.

Could you maybe post it here or put it somewhere to download?

Thanks a million!

Mike

[Fourty2]

any chance that you could make your dongle.bin available for the people that are less gifted in the use of compilers?
I also have a H3 that doesn't boot the current VOMP.

Could you maybe post it here or put it somewhere to download?

Get it here (CVS-Version with Wake-On-LAN - needs CVS-Vompserver):
http://www.hachmann-it.de/VDR/Dongle-CVS-WOL.ZIP


Anyone with a downloadable VompClient.exe out there?


Fourty2

[Fourty2]

One single question (looking into my server-logs...):

Into which difficulty do you run by just clicking a single link to a dongle-file?


Fourty2

[meando]

One single question (looking into my server-logs...):
Into which difficulty do you run by just clicking a single link to a dongle-file?

Err... none? I downloaded it and it works fine. Maybe you see search engine crawlers downloading it as well?

Anyway, thanks a million for the dongle.bin! Have seen H3 connect to VDR for the first time now. Works a great 80 percent with the debianized 0.2.4 server plugin (does not edit timers and does not play recordings but streams live tv just fine!). Guess I have to get and compile the server plugin from CVS...

Unfortunately, I could not make the MVP H3 bootp/TFTP for the dongle.bin from VDR. I used the trick to place your new dongle.bin (thanks again!) on the windows box with the hauppauge software, then power up MVP, let MVP get dongle.bin from windows and then switch off the windows server again...

Any clues on what I can do to make H3 request a dongle.bin from VDR?

Cheers,

Mike