Loggytronic Forum

VOMP => VOMP General / MVP => Topic started by: dingo35 on February 04, 2010, 09:06:35

Title: Bug vompserver on certain vdr-1.7.11 streams
Post by: dingo35 on February 04, 2010, 09:06:35
I have come across two different bugs in vompserver:
1) sometimes, when I start a recording, the current time/total time is not filled in correctly, but displays  --/-- .
When it goes wrong, the client sends command 7 directly after selecting the recording:
09:47:21.133495 [debug]  RecPlayer - FILENAME: /media/video0/NED2/2010-02-03.14.57.2-0.rec/00001.ts
09:47:21.133561 [debug]  RecPlayer - File 1 found, totalLength now 130461284, numFrames = 6599
09:47:21.133599 [debug]  RecPlayer - FILENAME: /media/video0/NED2/2010-02-03.14.57.2-0.rec/00002.ts
09:47:21.133686 [debug]  RRProc - written totalLength
09:47:21.133725 [debug]  RRProc - threadMethod waiting
09:47:21.134574 [debug]  Client - Received chan=1, ser=9808, op=7, edl=12
09:47:21.134611 [debug]  RRProc - recvReq set req and signalled
09:47:21.134632 [debug]  Client - Waiting
09:47:21.134657 [debug]  RRProc - thread woken with req, queue size: 1
09:47:21.134677 [debug]  RRProc - thread while
09:47:21.134702 [debug]  RRProc - received command 7
09:47:21.134723 [debug]  RRProc - getblock pos = 0 length = 10000


And when it goes right, it first does command 16:
09:47:21.453342 [debug]  RecPlayer - FILENAME: /media/video0/NED2/2010-02-03.14.57.2-0.rec/00001.ts
09:47:21.453409 [debug]  RecPlayer - File 1 found, totalLength now 130461284, numFrames = 6599
09:47:21.453448 [debug]  RecPlayer - FILENAME: /media/video0/NED2/2010-02-03.14.57.2-0.rec/00002.ts
09:47:21.453532 [debug]  RRProc - written totalLength
09:47:21.453572 [debug]  RRProc - threadMethod waiting
09:47:21.460124 [debug]  Client - Received chan=1, ser=9814, op=16, edl=4
09:47:21.460168 [debug]  RRProc - recvReq set req and signalled
09:47:21.460189 [debug]  Client - Waiting
09:47:21.460216 [debug]  RRProc - thread woken with req, queue size: 1
09:47:21.460237 [debug]  RRProc - thread while
09:47:21.460261 [debug]  RRProc - received command 16
09:47:21.460300 [debug]  RRProc - Wrote posFromFrameNum reply to client


2) ONLY on recordings from transponder 12515H on S19.2E (Ned1, Ned2, Ned3), there are positioning problems.
-When you are looking to a recording, the "seconds" field updates much to slow (so it takes 1,5 seconds to show 0.01)
-When fastforward or rewind, the point from which is jump is "back in time". Here is the log when I jump forward 10 seconds:
10:00:35.091598 [debug]  RRProc - received command 7
10:00:35.091621 [debug]  RRProc - getblock pos = 17550000 length = 100000
10:00:35.091928 [debug]  RRProc - written 100000
10:00:35.174426 [debug]  RRProc - Finished getblock, have sent 100012
10:00:35.174549 [debug]  RRProc - threadMethod waiting
10:00:35.256136 [debug]  Client - Received chan=1, ser=10804, op=16, edl=4
10:00:35.256265 [debug]  RRProc - recvReq set req and signalled
10:00:35.256289 [debug]  Client - Waiting
10:00:35.256321 [debug]  RRProc - thread woken with req, queue size: 1
10:00:35.256344 [debug]  RRProc - thread while
10:00:35.256371 [debug]  RRProc - received command 16
10:00:35.256431 [debug]  RRProc - Wrote posFromFrameNum reply to client: position=10600756 framenumber=485
10:00:35.256464 [debug]  RRProc - threadMethod waiting
10:00:35.257444 [debug]  Client - Received chan=1, ser=10805, op=7, edl=12
10:00:35.257478 [debug]  RRProc - recvReq set req and signalled
10:00:35.257498 [debug]  Client - Waiting
10:00:35.257522 [debug]  RRProc - thread woken with req, queue size: 1
10:00:35.257542 [debug]  RRProc - thread while
10:00:35.257562 [debug]  RRProc - received command 7
10:00:35.257631 [debug]  RRProc - getblock pos = 10600756 length = 100000


These particular stream are OK when live viewing, and also in VDR itself. I noticed that the "resume" function works correctly, so it has to do with the way vompclientrrproc.c derives the framenumber from the stream through ntohl. I cannot detect where this req->data is filled, though ...

A temporary fix is in line 1222 of vompclientrrproc.c:

  //ULONG frameNumber = ntohl(*(ULONG*)req->data); //FIXME DINGO
  frameNumber = x.recplayer->frameNumberFromPosition(x.recplayer->getLastPosition());


..but this isnt really ok since the last position is only a rough estimate, and is not updated frequently.

Can someone (Marten?) point me into the right direction: which fields in the ts-stream are causing this problem, and/or where is this req->data filled, and/or how to solve this in vomp (if VDR is not the right place to solve this) ?

Thanks Dingo.

Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 04, 2010, 16:13:22
Hello,
maybe you can give me test short samples, then I can trace the problem.


Marten
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: dingo35 on February 04, 2010, 18:14:10
You have mail...
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 05, 2010, 02:23:45
I have an idea, where the problem could be at 2).
The  DemuxerTS uses findPictHeader in order to count the frames. It assumes, that there is one or zero picture header in a pes. But maybe there are two in this channel.
Since vdr does not split up pes packets, if there a two in one pes packet, this might be the problem.
Can you check this?

Marten
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 05, 2010, 03:10:28
Now the second idea, problem number 1) is probably vdr new genindex function, that regenerates the index file every playback. This is a new function but leads to not having the data for seeking at playback start.
I will fix this on the weekend.

Marten
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 05, 2010, 05:04:18
With 1) I am not sure any more since I got this problem also with old vdr pes recordings. I have the weird behaviour that it happens for every recording only once, can you confirm this?
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: dingo35 on February 05, 2010, 08:16:56
No, that must be different problem: I have "my" problem _never_ with PES recordings, and _only_ with TS recordings from this particular transponder. It also happens repeatedly in such a recording.

I have a few minutes of sample stream here http://rapidshare.com/files/346156998/wrongts.tgz (http://rapidshare.com/files/346156998/wrongts.tgz)
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 07, 2010, 00:04:42
So problem 2) is solved, it is the assumption in current code, that there is only one frame per pes frame. (For the first one I am still investigating).
Which is not true for this channel.

It can be solved with adding:

UINT PESPacket::countPictureHeaders(/*bool h264*/) const
{
  if (size < 12) return 0;
  UINT pattern = ( ((UINT)data[ 8] << 24) |
                   ((UINT)data[ 9] << 16) |
                   ((UINT)data[10] <<  8) |
                    (UINT)data[11]  );
  UINT pos = 11;
  UINT count=0;
  if (/*h264*/ false) {
    
     while (pos<size)
     {
          pos++;
        pattern = (pattern << 8) | data[pos];
          if (pattern==/*DEMUXER_H264_ACCESS_UNIT*/0) count++;
     }
     return count;
  } else {
      while (pos<size)
     {
          pos++;
        pattern = (pattern << 8) | data[pos];
          if (pattern==DEMUXER_PIC_HEAD) count++;
     }
     return count;
  }
}


and you have to change the beginning ofvoid DemuxerTS::parseTSPacketDetails(PESPacket &packet) // Only important stuff for paket counting reminas
{
    parsePacketDetails(packet);
    if (packetCounting && packet.getPacketType() >= PESTYPE_AUD0 &&
                        packet.getPacketType() <= PESTYPE_AUDMAX)
    {
        packetNumber++;
    }
    UINT numpicts=packet.countPictureHeaders(/*h264*/);

  if (frameCounting && numpicts &&
      packet.getPacketType() >= PESTYPE_VID0 &&
      packet.getPacketType() <= PESTYPE_VIDMAX)
  {
    frameNumber+=numpicts;
    ULONG frame_num = frameNumber;

@Chris no neeed to include this in cvs now, I will provide apatch after the my hdtv changes are in cvs.

Marten
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: dingo35 on February 07, 2010, 10:23:29
Thanks Marten for your quick solution !!!!!
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 07, 2010, 21:40:58
So I tried to reproduce 1), but I was not able to do so.
This can mean the following:
1) Issue is mvp specific (I did it on windows, I have no mvp at hand oin the moment).
2) Issue is already addressed in the changes I made for h264 playback.
3) Issue depends on the timing between server and client, so that my configuration did not show this problem.
4) The fix for problem 2) solved the problem.

So I am sorry, if you can generate mvp builds yourself, generate a version of the dongle with more debug output(Log) in Player::threadFeedPlay(), I am sure that if the return values of the function, we might be able to get closer to the problem.

Marten
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: dingo35 on February 08, 2010, 14:36:57
Hi Marten,

Glad to report that your patch fixed problem 2) beautifully!! First tests show jumping + updating of seconds counter ok :-)

Problem 2) still exists, I will try to produce more output, it is hard to reproduce. I suspect it has to do with timings, I also have posted other patches that no-one seem to need, I have a videoserver with 4 DVB-cards, 3 client mvps and 4 windows-mvp,s so sometimes that generates a lot of load....

My Windows-compiler broke, could you perhaps post a windows version with your patch in it?

Thanks! Dingo.

Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 08, 2010, 15:20:21
My Windows-compiler broke, could you perhaps post a windows version with your patch in it?

No this is not possible in the Moment, I have in my development environment some sort of intermediate version for hd playback, which is incompatible to current cvs protocoll and would need to change this back and this is too dangerous for me in moment. I fear loosing some changes,


Marten
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 21, 2010, 22:47:10
I found now something, which might relate to the other problem.
At certain startup timings, the pts reported by the windows port was invalid, which caused the pts_map algorithm in demuxerts to flush the pts_map, which is needed for navigation.
Similar things may occur also in the mvp.

Marten
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: dingo35 on February 22, 2010, 08:13:43
Ok, I will test your HD version asap (hoping this issue will be fixed in that version :-)  ).

Thanks Marten!
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 22, 2010, 15:13:20
Wait, in the moment this fix is not online in the HDTV version, I will generate later a newer build.

Edit: Now it is online

Marten
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: hondansx on February 23, 2010, 18:52:38
Quote from: MartenR on February 22, 2010, 15:13:20
Wait, in the moment this fix is not online in the HDTV version, I will generate later a newer build.

Edit: Now it is online

Marten

Is there any patch for testing?

Thanks
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: dingo35 on February 23, 2010, 21:50:06
I found newest client: http://forum.loggytronic.com/index.php?action=dlattach;topic=507.0;attach=188 (http://forum.loggytronic.com/index.php?action=dlattach;topic=507.0;attach=188).

Shall I generate patch to update Yaris version or will you do that?
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 24, 2010, 02:47:51
The following patches are not in cvs (but should appear there finally):
1) Patch file attached. (Includes the patch reported earlier).
2) For the second problem, but it lacks something for the mvp, which sets the pts to zero, if nonsense is reported during startup: (I have currently no mvp availiable)
ULLONG VideoWin::getCurrentTimestamp()
{
   REFERENCE_TIME startoffset;
   REFERENCE_TIME ncr_time;
  if (iframemode) return 0; //Not in iframe mode!
  if (!dsrefclock || !sourcefilter) return 0;
   FILTER_STATE state;
   sourcefilter->GetState(10,&state);

   if (state==State_Running) dsrefclock->GetTime(&cr_time);
   ncr_time=cr_time;
  startoffset=sourcefilter->getStartOffset();
    if (startoffset==0) return 0;
   ncr_time-=startoffset;
   ncr_time-=lastreftimeRT;
/* ULLONG result=frameNumberToTimecode(
    VDR::getInstance()->frameNumberFromPosition(lastreftimeBYTE));*/
   long long result=lastreftimePTS;
   result+=(ULLONG)(ncr_time/10000LL*90LL);

    if (result<0) result=(1LL << 33)-result;
  return result;

}
The problem here was, that during startup nonsense values occurred, so that the pts_map in demuxerts was flushed. In this case the function returns zero and demuxerts will skip it, if called with zero:

ULONG DemuxerTS::getFrameNumFromPTS(ULLONG pts)
{
  ULLONG difference = (1LL<<33);
  ULONG ref_frame = 0;
  int total = 0, actual = 0;
  if (pts==0) return 0; //we are in startup


Marten
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: MartenR on February 24, 2010, 02:55:18
Maybe I should clarify what the problem is, if there is any pts value which makes no sense, it probaly will flush the pts_map in the demuxer. So maybe it has to be fixed there and I am only fixing synptoms.

Maybe Mark or Chris have an better idea.

Marten
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: hondansx on February 24, 2010, 20:26:46
Quote from: dingo35 on February 23, 2010, 21:50:06
I found newest client: http://forum.loggytronic.com/index.php?action=dlattach;topic=507.0;attach=188 (http://forum.loggytronic.com/index.php?action=dlattach;topic=507.0;attach=188).

Shall I generate patch to update Yaris version or will you do that?

Hi Dingo,

would be nice, if you could.
BTW the URL you posted includes only a exe file.
Title: Re: Bug vompserver on certain vdr-1.7.11 streams
Post by: dingo35 on February 25, 2010, 11:08:25
Exe file is from Marten ... updates on Yaris version in other thread.