6 07 2010

Experiment #2: Emulating protected members in JavaScript

Summary : Do you still believe that there are only public, private and privileged members in JavaScript objects? With JavaScript, you never can be sure.

Lego principle

As I might have told you, I think that JavaScript is an amazing language. Even so, I would like to repeat it again. It is the ultimate language for me, and I hope it’s so for you. JavaScript is like a Lego constructor, depending on how you combine different parts, you might get a car, a house or maybe just a pile of crap. Your imagination defines the rules.

This time I’ve discovered some unbelievable combination. It allows you to create and use so called “protected” members in JavaScript. How do I define protected members in JavaScript? As a mixture of both, private and public members, which gives you the possibility to inherit, override and access private members without use of a privileged bridge. If you don’t know what these members are or have just forgotten them, here is a link to refresh your memory. For now, let’s look at an example below :

function Some(...) {
    // defining private member
    var value = 'value';
}

Some.prototype.getValue = function() {
    return value;
};

Code : Scoping issue when adding methods to prototype

As you might have noticed, the code above is broken. Executing getValue function, will throw an exception, because the declaration of getValue is not in the same scope where value variable is declared. To make the code work, function getValue should be declared inside a constructor, hereby becoming a privileged member. This limitation makes it impossible to access the private variables outside the constructor.

The experiment I wrote does allow you to work with the private members outside the constructor, meaning you can add functions to the prototype and expect them to behave as if they are all in the same scope. Sounds cool, right? Actually, it doesn’t look half bad.

Some = Declare({
    _value : 'value',
    getValue : function() {
        return _value;
    }
});

SubSome = Declare( Some, {
    setValue : function(val){
        _value = val;
    }
});

Code : Declaring class with protected member using prodec wrapper

In the example above there is one protected member _value. It is declared like other public members getValue and setValue, but it is accessed as a private member. Accessing _value variable in the SubSome class shows that the private scope, indeed, “can be” shared. I would like to put extra emphasis on the fact that the protected member is not accessible from outside the object, there is just NO possibility to do that. This means that someObj._value will be unavailable.

I used a declaration wrapper to hide all implementation details ( I love code separation ), but here is the approximately unwrapped code, which will help you to easily “diff” both listings:

Some = Declare();

Some.prototype = {
    _value    : 'value',
    getValue : function() {
        return _value;
    }
}

//Adding inheritance
SubSome = Declare();
SubSome.prototype.__proto__ = Some.prototype; // Note: IE does not support __proto__ convention
SubSome.prototype.setValue = function(val) {
    _value = val;
}

Code : Declaring protected members with prototype

The code above does the same thing as the first broken example, except that it also consists of SubSome inheritance. The SubSome inheritance is there to show you that you can access protected members declared in a superclass. Consider that the constructor is created by the Declare function and not like usually by function(){}. ( For those who need explanation about __proto__- here is link ) Note, that it’s also possible to share protected functions, not only variables, like in this example.

Internals

I guess you already have plenty of questions concerning the implementation of this experiment. I will not withhold the answers from you and will start explaining all the details right now. If you think that the code is the best source for explanation, I suggest you move at once to the last part of this article or visit a playground.

The first answer is with :). I was always wondering how private scope could be shared. We have the ability to share “this” scope, by saving it as variable, but we have no way to save a private scope and unroll it somewhere else. In context of “protected” members, we really need to save the private scope and apply it in different places. Even though we can’t do that “officially”, it’s possible to emulate this behavior by using with keyword. I’m sure everyone of you know what with is doing, but everyone is avoiding it, because it’s a bad coding practice. Anyhow, let’s leave prejudice out of this and look at the possibilities with brings us.

var obj = { zork : "Bork" }
with (obj) {
    alert( zork );
}

Code : “With” Nature

This code, is doing exactly what is needed for emulating protected members. It allows you to access obj variables as if they are private variables. Looking from with perspective, the given obj can be seen as private scope. But let’s leave this part for a second and look at the next code.

var s = function(){
    var zork = 'Bork';
    return function() {
        alert( zork );
    }();
}
var fnc = s();
fnc(); // alerts Bork

Code : Closure nature

We know that internal closure bla will popup “Bork”, because of the nature of the closures. Now, I will combine both characteristics of closure and with and show what would happen if we create closure inside with. All the variables of obj will be available to the closure, meaning we could manipulate the scope :

var s = function(){
    var obj = { zork : "Bork" }
    with (obj) {
        return function() {
            alert( zork );
        }();
    }
}

var fnc = s();
fnc(); // alerts Bork;

Code : Combining “closure” with “with”

The result is the same, except now, we have our private scope coming from object and not from “var” declaration. This is really the main element of my experiment : Creation of the functions inside with and assigning them to the object. All thanks to the dynamic ability of JavaScript, which allows us to create and bind functions at runtime. Time to see how to apply this closure-with combination to create objects based on their prototype.

We know, that every constructor consists of a prototype object. This object contains almost the full definition of the object. I say almost, because functions can be created inside the constructor, which are not visible through the prototype. Even then, during object creation, it’s possible to walk through the prototype of the object and do things based on some requirements. For example : As you’ve already seen, protected members in my code start with an underscore. Based on this, we can collect all such members and place them inside our protected object. Later we can use this object inside with, so that the underscore members appear as private members.

function Some() {
    var proto = Some.prototype;

    this._protected = {};

    for (var i in proto) {
        if ( /^_/.test(i)) {
            this._protected[i] = proto[i];
        }
    }
}

Code : Creating protected scope object

Scope is built, it’s time to use it :

Some.prototype._value    = 'value';
Some.prototype.getValue = function() {
    with (this._protected) {
        alert( _value );
    }
}

Code : Using protected object to emulate private scope

As you can see, we’ve made the first, yet the most important step of the “protected members” principle. However, there are still a number of issues. To start with, protected scope is available to the outside world, because it’s publicly defined. Further more, it is the requirement for the with keyword in every function where protected scope is presented. These issues leave an implementation footprint in the declaration of the class, which does not look very neat. Both issues are related, meaning that fixing them should be done at the same time.

First of all, we need the technique to hide our protected scope from the public “eyes”. A solution to this would be to use the nature of private members. The protected scope object should be defined as a private member. The only issue in this case that we will be facing the problem described above, namely inability to access private members outside the constructor. A fix for that would be to keep creation of the functions inside the constructor as well. The question is, how? Well, in the constructor we have access to any function and to the object itself, so if we could clone every function of Some object and reassign it, the desired behavior would be achieved.

To clone a function I have to resort to using another “evil” keyword – eval. Because every function exposes its body via toString(), we could evaluate it, at the places we need. Here is clone example :

var fnc = function() { alert('a') ; }
var member = '';
var clone;
clone = eval( "clone = " + fnc.toString() );

Code : Cloning function

Because eval remembers the scope in which it was evaluated ( has access to the member variable), a function can be cloned inside the with statement, having access to the protected scope object. Well, enough talk, it’s time to put all the building blocks together and see the result :

function Some() {
    var proto = Some.prototype;

    var protscope = {};

    for (var i in proto) {
        var member = proto[i];
        if (typeof member == "function") {
            with (protscope) { //adding protected scope
                // cloning,
                // cloned function receives automatically the scope correct scope
                member = eval( "member = " + member.toString() );
            }
        }

        // doing reassign
        this[i] = member;

        if ( /^_/.test(i)) {
            protscope[i] = proto[i];
            // removing protected variables from outside
            this[i] = null;
        }
    }
}

Some.prototype._value = 'value';
Some.prototype.getValue = function() {
    alert( _value );
}
Some.prototype.setValue = function(val) {
    _value = val;
}

Code : Result of all the effort – creating protected members

If you are familiar with JavaScript the code should be fairly self explanatory. To get a better feeling for protected members, as I mentioned earlier,I’ve created a playground for you. I did not cover all parts of the declaration wrapper here, such as dealing with this keyword, using super classes, etc. If you are interested, grab the src-code and enjoy.

How much does it cost ?

What actually is a price of protected members? First of all – Speed. I haven’t performed any benchmarks, but by analyzing the code I can tell you that this emulation will run slower than normal creation of objects. Using with together with eval for every object member will stall the runtime. Secondly – memory consumption. Using a set of functions for every instance will eat up more memory in comparison to the prototype based instances.

Nevertheless, if the speed does not play a big role and you see advantages in using protected members you could adopt the src-code for your own needs.

classes, execution, experiment, javascript, protected, scope

IE9, Windows XP and 2014? Happy programmers’ day

17 thoughts on “Experiment #2: Emulating protected members in JavaScript”

Tweets that mention Experiment #2: Emulating protected members in JavaScript | Notes of Maks Nemisj -- Topsy.com says:

July 7, 2010 at 4:27 pm

[…] This post was mentioned on Twitter by Michiel Kalkman. Michiel Kalkman said: RT @maksnemisj: Finished "Emulating protected members in JavaScript" http://dzone.com/6jf6 […]
Sebastian Rostock says:

July 13, 2010 at 2:49 pm

Some time ago I developed the “xpc” library, that also supports protected members.

The name “xpc” stands for eXtensible Prototype Class and identifies the core of an object-oriented-multiple-inheritance-library for JavaScript. This library provides a comfortable interface for defining new classes. Noteworthy are the abilities to define classes and objects with private, protected and public fields and methods.

http://home.arcor.de/sebastian.rostock/modules/xpc-6.js

http://home.arcor.de/sebastian.rostock/modules/xpc-6.html

http://home.arcor.de/sebastian.rostock/modules/xpc-6.jsdoc
Sebastian Rostock says:

July 13, 2010 at 3:05 pm

I missed to state, that the generated classes are also very fast. There is an inner class for the protected members so the time needed to create an instanc is twice the time needed for a normal object plus the time for publishing of protected members.

a speed tester is here: http://home.arcor.de/sebastian.rostock/applications/jss/index.html
Maks Nemisj says:

July 13, 2010 at 3:22 pm

@Sebastian:

Hi, I looked at your code. Interesting idea, but I could not find the possibility to declare protected members. I used this code :
```
Super = xpc.Of(function(){
  var _value = 'initial';

  this.Publish('setValue', function(value){
    _value = value;
    return this.This;
  });

  this.Publish('getValue' , function(){
     return _value;
  });

  return function(){};
}); 

Sub = xpc.Of(function(){
   this.Super(Super);

  this.Override('getValue', function(value){
    alert('Current value is' + _value);
    return this.Super(value);
  });
  
  return function() {
    this.Super();
  };
});
```
The problem is, when you try to call getValue of the Sub class, it will throw error, due to the fact that _value is not visible there.

The only possible way to make it visible is to declare it public. The problem is, that if I do that, everyone will have access to it. This is the reason why protected members in my solution are all accessed as private members ( to make sure no one screw them ). Unless you cant’ create hidden member for outside but visible for inside ( like Sub should have ) you can’t call members protected.
Sebastian says:

July 14, 2010 at 6:35 pm

@Maks Nemisj

Hi, Maks Nemisj. First of all, I thank you for trying and commenting my work.
The idea behind the class creation via “xpc” is, that you define two classes at once, where both classes define public members. When creating a new object, one class will deliver the public members, the other one the protected ones.
So to access a protected member you have to use the protected object “this” inside the protected methods.The public methods are here accessed on the public instance “this.This” .

<

To make your code work, try this
```
Super = xpc.Of(function(){
  // here is the scope of the class-definition
  // (private static)
  // "this" is the protected class' prototype (protected)
  // "this.This" is the public class' prototype (public)
  // "this.Class" is the public class (public static)

  this.value = 'initial';

  this.Publish('setValue', function(value){
    // "this" is the protected instance
    // "this.This" is the public instance
    this.value = value;
    return this.This;
  });

  this.Publish('getValue' , function(){
     return this.value;
  });

  return function(){
    // here "this" is the protected instance
  };
});

Sub = xpc.Of(function(){
    this.Super(Super);

  this.Override('getValue', function(value){
    // "this" is the same like in the overriden method
    alert('Current value is' + this.value);
    return this.Super(value);
  });

  return function() {
    this.Super();
  };
});
```
Sebastian Rostock says:

July 14, 2010 at 7:16 pm

I missed my last name

… to instanciate the classes use

var o1=new Super();
var o2=new Sub();

because class-prototyps are used the instanciations are very fast.

One could access Super.prototype th manipulate the public members defined from the class Super, but this will not change this members in the ancestor class Sub.
To recreate Sub one couls use Sub=xpc.Of(Sub.constructor) because the function to define the classes and that was passed to xpc.Of bocemes the constructor of a class.
Sebastian Rostock says:

July 14, 2010 at 9:52 pm

… it is late and my english suffers on this. What i wanted to state is:

One could access “Super.prototype” to manipulate the public members defined in the class definition of “Super”, but this will not change the inherited members in the class “Sub”. To update “Sub” one couls write “Sub = xpc.Of(Sub.constructor)”. The function used to define a xpc-class bocemes the “constructor” of that class.
Maks Nemisj says:

July 15, 2010 at 8:47 am

@Sebastian Rostock

I tried your example and it works nicely . I like the way you define the protected members in instances. Good work.

P.S: Sometimes WordPress places your comments in a “to approve” list that’s why you have not seen them.
Sebastian Rostock says:

July 15, 2010 at 11:14 am

@Maks Nemisj

I thank you for repairing the mess I made in last the comments.
It would be nice of you, if you could tell me which parts of the documentation for the “xpc” should be explained better/deeper.
Sebastian Rostock says:

July 16, 2010 at 11:48 pm

@Maks Nemisj

To speed up your approach, you could use a mixture of your core work with the “with(…)” part and the class-constructor of “xpx” or at least its “invoke” part. The following code needs additional modifications, but in principle should work.
The creation time for new instances is then:
2 x Object + 1 x Function.
```
function Declare2(){

    // inner class for protected members
    var coreClass=null;

    // inner class for public members
    var thisClass=null;

    // outer class for defining members via prototype
    //   (name format public/_protected like in "Declare")
    var baseClass=function(){
        // create protected and public instances (fast)
        return new thisClass(new coreClass());
    };

    // creates a proxy/replacement for a protected method ("with(...)")
    var copyCore=function(method){
        return new Function('with(this){ with($this){ return ('+method.toString()+').apply($this,arguments); } }');
    };

    // creates a proxy/replacement for a public method
    var copyThis=function(name){
        return function(){
            return this.$invoke(name,arguments);
        };
    };

    // this map stores the public methods
    var invokeMap=null;

    // compile the inner classes
    baseClass.compile=function(){

        // process "arguments" like in "Declare"
        // define protected class
        coreClass=function(){};

        // define public class
        thisClass = function(core){

            // protected instance knows public instance
            core.$this=this;

            // call public method with protected scope
            this.$invoke = function(name,args){
                return invokeMap[name].apply(core,args);
            };
        };

        invokeMap = {};

        for(var x in baseClass.prototype){
            // copy protected field definitions
            if( "x is protected field") coreClass.prototype[x] = baseClass.prototype[x];

            // copy/convert protected method definitions
            if( "x is public method" ) coreClass.prototype[x] = copyCore(baseClass.prototype[i]);

            // copy public field definitions
            if( "x is public field" ) thisClass.prototype[x] = baseClass.prototype[x];

            // copy/convert public method definitions
            if( "x is public method" ){
                thisClass.prototype[x] = copyThis(x);
                invokeMap[x] = copyCore(baseClass.prototype[i]);
            }
        }

        return baseClass;

    };

    // initial update the hidden classes
    return baseClass.compile.apply(null,arguments);
}
```
Maks Nemisj says:

July 22, 2010 at 11:54 am

@Sebastian Rostock

Excellent idea, to rebuild prototype functions with inlined “with” 🙂
Sebastian Rostock says:

July 25, 2010 at 1:08 pm

@Maks Nemisj

I have rebuild the method and now it works fine. There is also an internal cache for the lazy created method proxies.
If one calls a public method, the call is forwarded via “invoke” to a private method in “map”.
The method in “map” then uses two “with” blocks to establish the protected and public scopes and calls the defined method.
If the prory does not exist, it will be created and stored on the protected instance.
This way protected methods can be called four times faster than public ones inside the methods of an instance.
```
function Declare2(){

    var coreCheck = /^_/;

    var coreClass = null; // inner class for protected members

    var thisClass = null; // inner class for public members

    var baseClass = function(){ // outer class for defining members via prototype (name format public/_protected like in "Declare")
        return new thisClass(new coreClass());// create protected and public instances
    };

    var coreProxy = function(name,method){ // proxy/replacement creator for protected methods (cached lazy "with(...)")
        return new Function('with(this){with(this["this"]){ return (this.' + name + '|| (this.' + name + '=' + method.toString()+')).apply(this,arguments);}}');
    };

    var thisProxy=function(name){ // proxy/replacement creator for a public methods
        return function(){ return this.invoke( name, arguments)  };
    };

    baseClass.compile = function(){ // compiler for inner classes

        var map     = {}, 
            coreMap = (coreClass = function(){}).prototype,
            thisMap = (thisClass = function(core){ 
                core['this'] = this;
                this.invoke = function(name,args){
                    return map[name].apply( core, args );
                };
            }).prototype,
            baseMap = baseClass.prototype, 
            next;

        for (var name in baseMap) {
            if ((typeof(next = baseMap[ name ])) == 'function') {
                if (!coreCheck.test( name )) {// convert public method definitions

                    map[name]     = coreProxy( name, next );
                    coreMap[name] = null; // initialize for cache
                    thisMap[name] = thisProxy( name );

                } else {
                    coreMap[name] = coreProxy( name, next ); // convert protected method definitions
                }
            } else {
                (coreCheck.test(name) ? coreMap : thisMap)[ name ] = next; // copy field definitions
            }
        };

        return baseClass;
    }

    var map  = baseClass.prototype, 
        list = arguments, 
        item;

    for (var i=0, size=list.length; i<size; i++) { // process arguments
        if ((typeof(item = list[i])) == "function") { 
            item = item.prototype;
        }

        if (item) { 
            for (var key in item) {
                map[key] = item[key];
            }
        }
    };

    return baseClass.compile(); // initial compile
}

var BaseClass = Declare2({
    _size   : 0,
    getSize : function(){
        return _size;
    },
    setSize : function(size){
        _size=size;
    }
});

var NextClass = Declare2( BaseClass , {
    incSize : function(){
        _size++;
    },
    toString : function(){
        return "NextClass("+_size+")";
    }
});

var next = NextClass();
alert(next);

next.setSize(3);
alert(next);

next.incSize();
alert(next);
```
Maks Nemisj says:

July 25, 2010 at 11:37 pm

@Sebastian Rostock

I like your optimization with lazy cache. It works indeed much faster. I’m really glad that protected scope emulation with “with”, gave you the optimization pleasure. This is what articles for, right? 🙂

I hope you do not mind, but I edit your code a bit, to make it more readable, I’ve added extra {} and extra spaces. You know, javascript is a powerful but you should not hesitate the keep it structured 🙂

P.S: if you want to instert code, use this tags <div><pre><code> code <closing tags>

Btw: You should try to use github, there you can branche my code and make your changes to it, thereby making it available to all people, and not only to comments 🙂
Sebastian Rostock says:

July 26, 2010 at 3:40 pm

@Maks Nemisj

Yes, it is fun to think through projects from some one else to find optimizations for them.

The given implementation in “Declare2” does the job, but the created objects are very expensive in means of memory usage. When I developed the “xpc”-core I also first defined the syntax I wanted to use for defining new classes. After that I tryed different approaches in the implementation and also found some memmory expensive ways. That is why the “xpc” library is already in version 6, but the current implementation is quite fast and doesn’t use that much memory. Currently every new instance of a “xpc”-class needs the memmory and creation time of two “Object” and one “Function”.

P.S: I will think about GitHub.

Btw: If you want, but you can already publish the results from this comment list into your GitHub account.
Emulating protected members in JS | RotterdamJS says:

July 26, 2010 at 4:24 pm

[…] Brush up on your JS and check it out. http://nemisj.com/protected-javascript/ […]
Mohammad Rahmani says:

April 15, 2016 at 7:37 pm

Excuse me , but what is Declare(), it is not defined at lease in the first few paragraphs and codes. Since I didn’t know what that is I had to stop reading this article.
Thank you
Maks Nemisj says:

April 17, 2016 at 8:48 pm

@Mohammad Rahmani,

Declare is the function exposed by the ‘prodec wrapper’. Library which I wrote to emulate protected members. You can find source code here https://github.com/nemisj/prodecjs

Experiment #2: Emulating protected members in JavaScript

Lego principle

Internals

How much does it cost ?

17 thoughts on “Experiment #2: Emulating protected members in JavaScript”

Leave a Reply Cancel reply