Work in progress

If you prefer to use an SVN client to download the code, point your client to the following address:

https://angelscript.svn.sourceforge.net/svnroot/angelscript/trunk

I recommend TortoiseSVN as the SVN client if you're using Windows, otherwise the original SVN is the best alternative.

Version 2.19.0 WIP - 2010/03/14

• Nothing yet

Known bugs

• 2009/01/25 - When compiling the library with option -O3 on Win32/MinGW the TestOptimize and TestConversion fails. status: reproduceable, needs fix
• 2009/05/27 - Portability issues with MinGW/GCC 4.x. status: needs investigation
• 2010/01/27 - A new version of MinGW fails to compile AngelScript due to missing strtod. This is a bug in MinGW, not AngelScript.
• 2010/02/14 - Possible bug on 64bit platforms. Seems related to enum type size.. status: needs investigation
• 2010/03/13 - Compiling the library with option -O3 on 64bit Linux causes errors.. status: needs investigation

Changes planned for later versions

These are some of the changes that I'm thinking about. Not all of these may actually be implemented, some may be implemented a bit differently than described, but all describe my current thoughts.

The list is ordered roughly by priority. The item I'll implement next depends mostly on the complexity on the feature, my current interest, and feedback from the user base.

You're always welcome to send me your comments on current and/or upcoming features.

version 3.0.0 requirements

• Remove the built-in array type. Arrays are supported as add-ons.
• Support for function pointers, so that the function binding through imports can be removed.
• Change script language parameter references to simplify syntax
• Addition of access profiles for individual registered types/functions
• Change to caller owns object handles.
• Separate compiler from the rest of the engine, so that it is possible to build applications without the compiler.
• Platform independent pre-compiled bytecode for compiling on one platform and executing on another.
• TypeId and asIObjectType should be removed. asITypeInfo should be introduced instead.
• function id should be removed. asIScriptFunction pointers should be used instead.
• Remove the syntax for declaring dynamic array types, i.e. int[] arr;. Dynamic arrays should be declared as the template type, e.g. array<int> arr;. Static arrays (when implemented) should still be declared with int arr[];

planned for next releases

• asEP_CALLER_OWNS_HANDLE
• Implement implicit factory/constructor behaviours for implicit cast from primitive type to object type.
• Allocate value types on the stack instead of on the heap.
  • Need to change how objects are returned by value in the scripts
• Allow scripts to return references. Then change the index behaviour to the opIndex method.
• Make the saved bytecode smaller and platform independent.
• Allow recompiling bytecode without destroying objects. In this mode, functions with the same function signature will simply have the bytecode updated, but will otherwise remain the same. There are obviously restrictions for this to work. Classes cannot change its properties, interfaces cannot change its members, global variables cannot change types, etc.

planned for 2.19.0

• Pragma callback in CScriptBuilder
• Look into MSVC 2008 project changes
• Add optional parameters to array constructor to initialize multi-dimensional arrays. The extra parameters will be ignored for arrays of lower dimension.
• Look into allowing difference in handle for get/set accessors, as long as the base type is the same. What if only get accessor is supplied? How should it be written to make the compiler understand that that the property should be treated as a value rather than a handle?
• Improve MinGW makefile
• Look into adding support for reutilizing contexts for nested script calls (suggestion by Thy Reaper)
• Improve the saved bytecode to decrease file size and to provide better portability
• Must be possible to register function definitions from the application, and also to enumerate them.

Library

Improved documentation

Some suggestions for improved documentation:

• Expand the pages on script classes. Show script example on how to use script classes (no need for 'new' keyword, temporary object, inheritance, polymorphing, const correctness)
• Show script example on how to use script arrays (how to resize, initialization lists, declaration, etc)
• Explain how function overloading works and how the compiler determines which function to call based on argument types.
• Explain how a script class can be passed to the application
• Add a page on the customization options. Preprocessor defines for compile time changes. Engine properties. Add-ons.
• Case study 1: using script classes to implement controllers for game entitites (can even use an abstraction layer to allow substituting the script controller for C++ dlls)
• Case study 2: using global script functions to implement the controllers
• Case study 3: implementing animation with co-routine
• Why use scripts: explain my view on why use scripting : safety, easily modified, remote debugging, etc
• Usage examples for asCALL_GENERIC
• Explain why the asBEHAVE_VALUE_CAST doesn't allow casts to bool.

Improved garbage collection

Some objects, such as script functions and global variables, will always be marked as long lived.

Improve Getters and Setters for properties

Allow the property accessors to access a real property of the same name without going into a loop.

Allow use of property accessors with indexes to wrap native C++ arrays. Suggestion by Bismuth

Allow use of compound assignment with property accessors, where possible. Forum thread. If the property is an object and the object has the compound assignment operator, what should be done then? For primitives we know what the result should be, but for object we do not so we cannot rewrite it to 'o = o + e', since the += operator may not do a simple addition.

More options for dynamic modules

I'm not quite sure how this will work, but it is something I would like to add.

Improved memory usage

Some other stuff that isn't necessary to keep around is the enums and typedefs that won't be used again unless other compilations are done.

Support smart pointers

Smart pointers is a class that holds a pointer to the real object. The class usually has the -> operator overloaded so that accessing members of the real object works just like accessing members through a normal pointer. The smart pointer may also take care of reference counting.

When registering the type controlled by a smart pointer it would be necessary to tell AngelScript that it is a smart pointer. A new special behaviour is necessary to register the -> operator so that the VM can automatically access the true object. When registering the members of the object it would also be necessary to indicate which members are accessed through the -> operator, and which are accessed directly.

How would it work to pass objects of this type to application functions, or return them from application functions? Usually objects held in smart pointers, are passed as normal references to application functions, but sometimes it really is a reference to the smart pointer itself.

It should be completely transparent to the scripts if an object is a smart pointer or not.

Custom operator tokens

Module access profiles

The access profile can be different for an object type and its methods, thus allowing modules to have access to different set of methods depending on their profile. The object's behaviours are always in the same access profile as the type itself though (or maybe not? It may be interesting to allow different profiles for factories, for example).

The config groups will still be removed as a whole.

The access profile should be set as a parameter when registering the type or function. The parameter should be optional (with value 0), and will by default add the type/function to all profiles. There will not be any Begin/End functions for setting access profiles. We may use an engine property to set the default access profile, if necessary.

The module's access shouldn't be set in the module, but rather be given as a flag to the compiler when the compilation is done.

User data

Suggestion by SiCrane

Improved multithread support

Allow applications to disable global variables in scripts through an engine property. This is useful if the application plan on having multiple threads execute scripts from the same module simultaneously and don't want to burden the script writer with having to think about how to control the access to the global variables to avoid memory corruption.

Multiple OS fibers

Currently if you have two contexts running in different fibers that switch between each others from within asCScriptContext::Execute, the asGetActiveContext() function may not return the correct context. In debug mode it may cause assert failures, as there is a check when popping the active context to make sure it is the correct one that is popped.

A switch of fibers from within asCScriptContext::Execute may also leave loose reference on the context stack if the application registered function that performs the switch is not properly implemented. To guarantee no loose references, the function should be a void function that takes no arguments. This way the script can only call it as a simple function call, rather than in a complex expression.

Optimize strings

The string constants used throughout the scripts should be instanciated once by the compiler and stored in a pool. The generated code will then receive constant references to these pooled strings. This will greatly improve performance for scripts that does a lot of string comparisons with constant strings, and will also permit application conversion of string constants from 8bit to 16bit for example.

Consider adding the primitive type 'text' which is basically a 'const char * char'. The string literals would be of this type, instead of the registered string type. When assigning one 'text' variable to another, only the pointer is copied, without any reference counting.

The 'text' type has some tempting advantages in that they are fast and efficient. They would also allow registration of C++ functions that take 'char *' as parameters. However, there are problems that needs to be addressed before taking this step, how should memory management be handled? What if the script stores a 'text' variable somewhere and then discard the module that compiled the text literal? The application writer could also be tempted to write functions that return 'text' literals to the script, how should this be handled?

Improved memory management for value types

Better validation of registered functions

Stack corruption happens for example when AngelScript passes a structure by value, but the C++ function expects a reference, or if the C++ function returns a value in memory, but the function was registered as void.

Can the calling convention be determined with means of templates? I.e. is it possible to use templates to determine whether a function should be asCALL_CDECL or asCALL_STDCALL? Example code from SiCrane

Add a way to obtain the function id of the system function being called

Registered comparison operator

If only the equality operator is registered then only the opEqual function will be supported. To be able to do all comparisons, one equality behaviour and one relational comparison behaviour is needed. Example: "less than" and "equal". The function will first compare with "less than", and if not less, it will compare with "equal".

All script code should be resumable

Some changes will be necessary to give the application complete control over script code that is executed. CreateScriptObject and Build will be able to use an application supplied context instead of creating its own. This will be done similar to how ExecuteString currently does it.

The script array object will also be modified to use a script function to perform the initialization of the elements. Script code can then call that script function directly instead of calling a C++ function that has to create a new context to initialize the elements. The asIScriptArray interface will of course maintain the C++ resize method, but it can internally call the same script function to initialize the elements, and permit the use of an application supplied context.

Improving context interface for calling script functions

When the argument is an object handle and takeOwnership is false, the context will automatically increase the reference count. If takeOwnership is true, the context won't increase the reference count, which means the application must not release it.

When the argument is an object by value, the context will automatically create a copy of the object, unless takeOwnership is true, in which case the context will store the object pointer and then release it upon completion.

Shared type ids

I need to expand this to other types, such as typedefs and enums, as well.

Classes are a bit more complicated, because they come attached with function implementation, which may access global variables, or other types that are not shared, etc.

Improved portability

• Improved support for complex types in native calling conventions on PPC

  PPC has very complex calling convetions and although AngelScript handles most cases well, it has problems when passing complex types around by value, either in parameters or as return value. For example a structure with only float values should be passed in the registers instead of in memory. Likewise, there is a vector type that has its own registers that AngelScript doesn't even know about yet.

  In order to be able to support functions natively (the generic interface already works) that pass these types around by value, AngelScript must know the exact layout of the structure. A change to the way types are registered is needed to support this, and the PPC native code must also be improved.

  A simple template helper function could be written that will allow values to be passed by value rather than by pointer. Otherwise you won't be able to call the function with a literal constant value.

• Native calling conventions on more compilers/platforms

  64bit Mac OS X is possibly the last major platform that AngelScript doesn't have support for native calling conventions on. I don't have access to a 64bit Mac OS X, so unless someone can contribute the code it will probably be a while before I get to this.

  Remember that although native calling conventions are not supported on all platforms the library still works by using the generic calling convention, so the support for native calling conventions is only for convenience/performance sake.

• Multithreading support on more platforms

  The multithreading support must be expanded to more platforms. Currently only Win32 and posix enabled systems can use multithreading.

• Indicator for non-trivial copy constructor

  The existance of a non-trivial copy constructor is often used to decide how an object is passed/returned by value. Will probably need to add something like asOBJ_APP_COPY_CONSTRUCT. Reference.

AngelScript calling convention

Objects parameters

All objects should be passed by reference, as if it was a const reference. The script functions that access object parameters for modification will copy the object first and then access the copy instead. Application registered functions that register parameters with non-const input reference should be wrapped automatically, by the compiler to pass a copy of the object to the parameter.

This will allow me to remove parameter references from the script language, without loosing the performance benefits of using const &in references.

Parameters can still be declared as const, but it won't make a difference to the caller, as it will always pass parameters expecting them not to be modified. This may affect function overloading.

Object handles

The caller should be the owner of the object handles passed to functions. This will reduce the number of addref/release calls made during script execution. It will also simplify the registered application functions, as they will no longer have to release the handles received in parameters.

The script compiler will need to make sure that all object handles passed to functions are stored in local variables (or parameters in the function). This is similar to how the compiler currently guarantees the validity of references.

In order to avoid abrupt changes to applications I'll implement this feature as an engine property: asEP_CALLER_OWNS_HANDLE. The value will be false by default. After a few releases I can change the default to true. Then after even more releases, the engine property can be removed all together.

Improved object type registration

Look into what to do with asOBJ_NOHANDLE and ALLOW_UNSAFE_REFERENCES. Currently the engine permits the types to be passed by reference. It probably shouldn't allow that, but is it necessary to block it?

Can we determine the asOBJ_APP flags through templates without loosing portability?

Cloning modules

The function ids for each module will be different, as a function id is a global id the uniquely identifies a function in the engine. The same goes for object types. This is especially important when the functions/classes accesses global variables in the module.

Interface ids will however remain the same for both modules.

A module can also be cloned by saving the bytecode from one module and then reloading it into another. But if the script engine can clone the modules directly it will be faster than having the application take care of it.

A common super class for all script classes

Potential drawback: Application registered classes won't always be derived from the super class, thus separating them from the script classes.

Advantage: All script classes will have a common denominator, facilitating the storage of them. An application registered class that derive from an AngelScript class interface, can be registered to have this same common denominator which should make it possible to have script classes inherit from application registered classes.

Evaluate the possibility of using dynamic calls

Investigate possibility of allowing reference cast behaviours for value types

Improve template classes

Remove built-in dynamic array objects

The built-in dynamic array type will be removed. This will happen when I release version 3.0.0.

Allow initialization lists for registered types

It must be possible to register types that accept initialization through initialization lists. This probably needs a new way of registration. Possibly a way to tell AngelScript which constructor should be used, and then which index behaviours.

It would be great if the application could tell AS if a special pattern is needed for the initialization lists, for example a map may need to be initialized with alternating types, e.g. {string, int, string, int}. Or perhaps {{string, int}, {string, int}}

Use copy constructor for initializations

More engine properties

• NO_64BIT_TYPES, which removes double, int64, uint64, etc.
• MAX_PORTABILITY, which removes support for native calling conventions.
• ALLOW_POINTERS, add support for pointers.
• NO_GLOBAL_STATEMENTS, which disables the use of global variables/statements in scripts. Default: FALSE.
• NO_IMPORTS, which disables the use of bound functions in modules. Default: TRUE.
• ALLOW_BOOL_CAST, which adds implicit conversion to/from bool
• REMOVE_LINE_CUES_BETWEEN_TRIVIAL_STATEMENTS, this does a slightly less radical optimization than BUILD_WITHOUT_LINE_CUES. SUSPEND instructions for statments that make calls to other functions or conditional branches will be kept to allow proper debugging. This one will be important for JIT compilation as it will allow longer sequences of bytecode to be transformed into machine code.

Allow importing of classes from other modules

Add support for importing class definitions from other modules, so that a class can be implemented in one module yet used in another.

Support for UTF16 encoded scripts

Adding a UTF-16 code scanner for the compiler might be interesting. This would allow applications to avoid having to convert script code to UTF-8 before compiling it.

This is very low priority though.

Automatic translation of the string type to other C++ built-in types

It will probably need a new behaviour on the string type, as well as a new indicator when registering the function that needs the translation.

Suggestion by Manu Evans.

Registration of true constants

Catch application exceptions when calling registered functions

Support for @+& (autohandles with reference to handle)

Suggestion by Dentoid.

Improved memory footprint

I've made great improvements on the memory footprint already, but at some time I will probably go back to this problem to see if I can improve it even further.

Statement blocks can be parsed one statement at a time, which will improve memory utilization when compiling large script functions.

Need engine properties to discard information that are not needed for runtime operation, i.e. enum values, type defs, local variable names, line numbers, etc.

Class methods registered as global functions

For singletons this can easily be done through wrappers as the object pointer can be resolved through a global variable or a static method to retrieve the singleton pointer.

Where the class method is not for a singleton it might be a little more difficult to solve with wrappers as it will be difficult to determine the object pointer. Because of this the best solution may actually be to implement this support in the core library after all.

Improve the way the compiler determines use of temporary variables

Separate compiler from VM

Rename asIScriptContext to asIScriptThread (or asIExecutor)

A better name may be asIExecutor, since the context really is that, an interface for executing script functions.

Context redesign

The context holds the environment in which threads are executed. Thus it will keep track of global variables and memory allocations, shared between threads.

A thread will only control the execution of a script, i.e. hold the call stack while it is running. Thus it will not be necessary to keep the call stack in memory when it is not used.

Unless, non-shared global variables are implemented, this redesign is pretty useless as the contexts will not hold anything.

Registerable context properties

A context property is a non-shared global variable registered from the application. After creating the context, the context property must also be set, otherwise a null pointer exception will occur.

I'm not sure if this is actually useful, so this will probably not be implemented.

Support debug only functions

dynamic inclusion of script code

Persistent script states

Compile to native code JIT

Precompiled bytecode

• Make the precompiled bytecode less dependent on the engine configuration

  It may be possible to make the precompiled bytecode less dependent on the engine configuration if we store more information while saving the bytecode. While loading the bytecode the engine should adjust it to adapt to the current engine configuration. This means that it shouldn't be a problem with adding new functions and global variables, reorganizing existing ones. Of course, if a function/variable that the script uses has been removed, or changed type, the load will fail. It may even be possible to have the loading adjust access to class members, though that is a more difficult.

  The advantage of this is that it would be easier to write a common script compiler, without having to share all the code with the application that will use the scripts.

• Platform independent bytecode

  With independent bytecode it would be possible to compile a script on one platform, save the bytecode, and the load the bytecode for execution on another platform.

  This is something that I want to do someday. Though I'm not sure how well it will work with AngelScript being so closely tied to C++. Pointers wouldn't work for example, as that is too low-level. The boolean type is also not equal on all platforms.

  Things that are currently making the code platform dependent:

  • endianess
  • pointer size
  • alignment/packing of class members
  • alignment of variables on the stack
  • size and value of boolean type

  Could perhaps have a an engine property to tell the compile to produce some platform independent intermediate bytecode, rather than the bytecode that the VM will execute. When loading this pre-compiled independent intermediate bytecode, the loader would need to convert it to the final platform dependent bytecode.

Allow script classes to inherit from application classes

Some info

Script language

Add support for final class methods, and final classes

Allow return of reference

The script language should allow this too, but it must take care to make sure the reference is valid.

• Must not be allowed to return reference to local variable
• Must not be allowed to return reference to parameter passed by value
• Must not be allowed to return reference to temporary object
• Must not be allowed to return reference returned from a class member of a local variable
• Must be allowed to return reference to global variable
• Must be allowed to return reference to class member
• Must be allowed to return reference to input reference parameter (only if the returned reference is const, as the input parameter is not supposed to be modified)
• Must be allowed to return reference returned from a global function
• Must be allowed to return reference returned from class member of a non-local variable
• Must be allowed to return reference to in/out parameter
• Should it be allowed to return reference to output parameter? The reference is to a temporary value so what would it mean?

Reflection

It is already possible to implement enumeration of types, variables, functions, etc, and to create instances of new classes.

It's possible, but complicated, to implement an 'invoke' function that calls an arbitrary function/method with arbitrary arguments. To support this I need to implement support for variable argument lists.

It's not yet possible to dynamically create new functions from within the scripts. For this to be possible the engine needs to be able to compile single functions in a module, and have support for function pointers.

Treat all reference types by handle

Modern languages such as D, Java, C#, and most script languages treat all reference types by handle, i.e. all variables of these types are only handles to instances, and all functions parameters are also by handle.

The one thing you lose with this is an explicit assignment operator for copying content as the assignment operator will perform a handle assignment rather than copying the content of the object. (Though it may be possible to create another operator for this.)

The advantage of implementing this would be a cleaner language syntax, in that the @ can be completely removed, thus also simplifying the compiler.

I haven't decided whether I really want to implement this, but I don't think it would be too difficult. If I decide to implement this, it would be for version 3.0.0 together with the change to the parameter references.

Note, AngelScript already supports implicit handle types by turning on the engine property asEP_ALLOW_IMPLICIT_HANDLE_TYPES.

Improved algorithm for finding best matching function

Initialize all variables to 0 by default

It would also be easier on the script writer I believe.

If I do this, I have to remember that variables should not just be initialized to zero when entering a new function. The variables must be initialized to zero every time a new scope is entered, e.g. each iteration of a loop.

This might be solved by using push and pop to handle the scope. When push is used to increase the stack space it will be incremented with zeroes.

I may be able to do this initialization only if the compiler discovers that the variable is used without being initialized first. That would avoid unecessary bytecode being generated, and thus not impact performance.

Improve function pointers

Taking the address of a function must be able to resolve function overloads automatically by checking the type of the variable where the address will be stored.

It must be possible to declare function definitions that use types declared below it in the script.

Implement try/catch statements in the scripts

Support for typeof() and sizeof()

Global statements

This also means that global variables will be initialized in the order they are declared, and that a global variable doesn't see variables declared after it.

Functions and class declarations will still be seen from above themselves. This also means that if a function is called from the top of the file, it can access variables that haven't been fully initialized yet (it will only see the default value). In case of accessing object variables, this will result in a null pointer exception.

Change script language references

See also: Parameter references

Improve multi-line strings

Static arrays

Allow use of static arrays in the scripts, e.g. int ar[3]. These will be directly compatible with C++ arrays. They will not be objects, so it will not be possible to store a handle for a static array, but they can be passed by reference to functions.

It should be possible to assign part of a bigger array to a smaller array, e.g.:

int a[10], b[3];
b = a[3..5];     // copies index 3, 4, and 5 to the array in b

Look at the D language for more ideas on arrays.

The static array types should be treated as an easy way to declare multiple variables that can be accessed through an index.

The static array type can have a size that is defined at run time, e.g. with a parameter name. In this case the memory for the array is allocated on the heap, plus a hidden variable that gives the size of the array. The same thing happens if the size of the array is fixed, but too large, e.g. larger than 100 elements.

The static array type will only be implemented when the dynamic array type has been removed from the language.

When passing a static array by parameter, or returning it, the size of the array must be fixed. That way it is not necessary to pass the size of the type as a separate variable.

default argument value

Ex: void func(int arg1, int arg2 = 0);

switch statements with string constants for case labels

Allow calling constructor from within constructor

Default initialization of class members

class MyClass
{
  int m_int = 123;
  string @m_str = @do_something();
}

// The integer functions can read 1,2,4, and 8 byte integers
int64  ReadSignedInt(uint byteSize);
uint64 ReadUnsignedInt(uint byteSize);
float  ReadFloat();
double ReadDouble();

void   SetEndianFormat(bool littleEndian);

// Returns true if successful
bool   WriteSignedInt(int64 value, uint byteSize);
bool   WriteUnsignedInt(uint64 value, uint byteSize);
bool   WriteFloat(float value);
bool   WriteDouble(float double);

asDECLARE_ARG_PUSHER2(arg_pusher, float, string);

// Generates
void arg_pusher(asIScriptContext *ctx, float a1, string a2)
{
  new(ctx->GetAddressOfArgLocation(0)) float(a1);
  new(ctx->GetAddressOfArgLocation(1)) string(a2);
}