You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now using idrefs in python-stix is somewhat clumsy. When working with related objects, it's often desirable to reference objects, rather than embedding (or inlining) them. However, doing so in python-stix often requires duplicate objects to be created when working with Related* classes:
Source: http://stixproject.github.io/idioms/indicator/c2-indicator/7stix_package=STIXPackage()
8ttp=TTP(title="C2 Behavior")
910indicator=Indicator(title="IP Address for known C2 Channel")
11indicator.add_indicator_type("IP Watchlist")
1213addr=Address(address_value="10.0.0.0", category=Address.CAT_IPV4)
14indicator.add_observable(addr)
15indicator.add_indicated_ttp(TTP(idref=ttp.id_))
1617stix_package.add_indicator(indicator)
18stix_package.add_ttp(ttp)
On line 15 we need to create a separate TTP instance and set its idref property to the id_ of the actual TTP definition. It is possible to do the following:
15indicator.add_indicated_ttp(ttp)
However, this will embed (or inline) the TTP definition, rather than reference it.
Proposed Solution
Add a by_reference flag for methods that establish relationships...
While, theoretically, this makes life easier for developers, it raises some issues:
If I iterate over the indicator.indicated_ttps, do I get back the fleshed out TTP definition or the skeleton TTP with the idref ?(indicator.indicated_ttps is actually a list of RelatedTTP instances..but you get what I'm saying)
Does this automatically insert the TTP definition into the STIXPackage.ttps list? (I don't think it should as referenced TTP definitions may exist elsewhere)
How do we handle the placement of the TTP definition (where does it actually go and at what point during runtime does it get there?)
How do we ensure a well-formed, schema-valid document will be produced (avoiding id collisions/duplicate constructs)?
It may be best to still require developers to place the TTP definition where it needs to go rather than try to figure out where it should go automagically:
This is a difficult problem. I tried to work on something similar in python-cybox with caches, and it works ok, but is a bit clumsy. When parsing XML, it depends entirely on the order of the parsing to know whether the "actual" item is in the cache when the "reference" is parsed. The "dereference" operation is a bit clumsy as well, and I haven't thought through it entirely yet. It's also possible to have circular references between items (if each is related to the other), which is another wrinkle in the parsing process.
I would lean towards "no".
For now, I think it makes the most sense for the author to explictly control where it goes, and only include references anywhere else. It's more than typical content authors should need to concern themselves with, but I can't think of a better way.
I've been thinking about a global "validate" function that gets called before items are exported, which could check this and other things that the validator currently does, and some it doesn't (i.e. making sure required elements are there, since the bindings don't enforce them). This could potentially be turned off in the to_xml() function (to_xml(validate=False)). Just a thought.
An alternative implementation (which I think I would prefer) is to add a "get_reference()" function to the core constructs, which could make the actual line:
indicator.add_indicated_ttp(ttp.get_reference())
You could also write a global utilty function that works on any object with id_ and idref properties:
Semantically there doesn't seem to be any difference between referencing an internal object and embedding it directly at that point in the tree, or am I mistaken?
We have begun working on a Cacheable mixin class that handles this, but our initial commits only resolve idref objects. This really needs to handle the coupling of idref and timestamp, so you can easily resolve an object with a given id at a defined timestamp.
Other considerations
Should we allow a user to pass in only an idref to resolve? What's the behavior when more than one object has that id with different timestamp values? Do we return a sequence of objects?
An approach to making this easier has been implemented in the mixbox branch of python-stix.
Classes that can be referenced via idref have filter() and get() classmethods, which allow fast lookup of instances by id_ and other properties (timestamp) if needed.
Activity
gtback commentedon Apr 4, 2014
To respond to your points:
to_xml(validate=False)). Just a thought.An alternative implementation (which I think I would prefer) is to add a "get_reference()" function to the core constructs, which could make the actual line:
You could also write a global utilty function that works on any object with
id_andidrefproperties:wbolster commentedon Nov 28, 2014
Semantically there doesn't seem to be any difference between referencing an internal object and embedding it directly at that point in the tree, or am I mistaken?
gtback commentedon Dec 1, 2014
@wbolster: In general, there is no semantic difference. There are a couple nuances, though, discussed here.
bworrell commentedon Apr 2, 2015
We have begun working on a
Cacheablemixin class that handles this, but our initial commits only resolveidrefobjects. This really needs to handle the coupling ofidrefandtimestamp, so you can easily resolve an object with a givenidat a definedtimestamp.Other considerations
idrefto resolve? What's the behavior when more than one object has thatidwith differenttimestampvalues? Do we return a sequence of objects?bworrell commentedon Jun 10, 2015
An approach to making this easier has been implemented in the
mixboxbranch of python-stix.Classes that can be referenced via
idrefhavefilter()andget()classmethods, which allow fast lookup of instances byid_and other properties (timestamp) if needed.