| Store | Cart

Re: Cannot write serialization code without access to Perl_hv_backreferences_p

From: demerphq <deme...@gmail.com>
Mon, 22 Dec 2014 15:51:49 +0100
On 22 December 2014 at 13:51, Aristotle Pagaltzis <paga...@gmx.de> wrote:

> * bulk88 <bul...@hotmail.com> [2014-12-21 23:30]:> > demerphq wrote:> > > Imagine I have a structure like this:> > >> > > my $x="foo";> > > my $r= \$x;> > > weaken($r);> > >> > > If we then serialize $r, when we deserialize it if we preserve the> > > weakref then instantly the deserialized $x will go out of scope.> >> > Then don't do a FREETMPS or LEAVE until the entire data on disk> > structure is in Perl space. Something like> >> > SAVETMPS;> > ......later on......> > sv_2mortal(SvREFCNT_inc_simple_NN(SvRV(sv)))> > sv_rvweaken(sv);> > ......later on......> > FREETMPS;>> You completely missed the point. The issue is not that $r will go out of> scope *during* deserialisation, but that it will go out of scope *after*> deserialisation – i.e. it will not be in the deserialised data structure> returned to the caller of the deserialiser. [^1]>> But should be.>> Which means that the deserialiser needs to know that all references to> it are weak, so it can strengthen one of them so that it will survive.>>
Thank you for clarifying my point here. You are quite correct.


> However, Yves:>> Can this really not be done without Perl_hv_backreferences_p? Namely,> since you are deserialising the structure, you know every part of the> structure. You also know when you are recreating a reference to some> other part of that structure, and whether that was a weak reference or> not. So you can just keep track of whether a referent has any strong> references or not – no? Am I missing something?>

I am not sure you are so much missing something as just not considering the
performance implications involved.

In order to do what you described Sereal would have to track *every* item
in the structure, on the off chance that one of the items was the referent
of the very rare weakref. Note, we have no guarantee that as we traverse
the data structure that we will encounter the weakref before the referent.

Whereas with something Perl_hv_backreferences_p() and the equivalent magic
logic, we can check every item before we serialize to see if it is the
referent in a weakref, if it is we then track how many times we see the
item, and where it is in our data structure.

So with one way we scale to the number of weak referents in our
datastructure (note that we might be serializing a weak referent, but not
the weakref itself), which is often 0, with the other we scale proportional
to the total number of items in our data structure.

BTW, Sereal does something similar with references as well. If during
serialization we see an item with a refcount larger than 1 we keep track of
where we serialized it. If later on we find a reference to it we then
output either a REFP or an ALIAS tag to reference the original.

Now, if Sereal were a two pass serializer like DDS we could do one
traversal over the data structure, and use the bookkeeping for that to
control the second pass. But that would obviously be inefficient as well.

So all together this is a performance issue. If a serialization tool wants
to handle weakrefs properly and efficiently it needs something like
hv_backreferences_p(). Having said that I have a patch that introduces a
new API call in yves/backrefs. I think probably hv_backreferences_p() can
be made internal again, and the use case replaced by the proposed
new sv_get_backrefs().

Cheers,
Yves

-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Recent Messages in this Thread
Father Chrysostomos Dec 21, 2014 08:49 pm
demerphq Dec 21, 2014 09:23 pm
demerphq Dec 21, 2014 09:24 pm
demerphq Dec 21, 2014 09:53 pm
bulk88 Dec 21, 2014 10:25 pm
demerphq Dec 21, 2014 10:31 pm
bulk88 Dec 21, 2014 11:31 pm
demerphq Dec 22, 2014 02:22 am
Aristotle Pagaltzis Dec 22, 2014 12:51 pm
demerphq Dec 22, 2014 02:51 pm
Steffen Mueller Dec 22, 2014 04:56 pm
demerphq Dec 22, 2014 05:06 pm
Aristotle Pagaltzis Dec 23, 2014 06:44 am
demerphq Dec 23, 2014 01:41 pm
bulk88 Dec 21, 2014 10:11 pm
demerphq Dec 21, 2014 10:15 pm
Steffen Mueller Dec 21, 2014 07:23 pm
Messages in this thread