Added option to XMLStream.send() to skip applying filters.
Filters in the out_sync group are synced with placing stanza content
either on the wire directly or into the send queue. Because of this,
out_sync filters should not block.
New stanza interfaces:
Adding a message receipt request:
msg['request_receipt'] = True
Adding a message receipt:
msg['receipt'] = '123-24234'
Retrieving the acked message ID:
ack_id = msg['receipt']
print(ack_id)
'123-24234'
New configuration options:
auto_ack:
If True, auto reply to messages that request receipts.
Defaults to True
auto_request:
If True, auto add receipt requests to appropriate outgoing
messages.
Defaults to False
Plugins that are referenced as dependencies, but have not been
registered now will be imported. Newer plugins should register
themselves automatically, but older style plugins will be
explicitly registered after import.
The new system is backward compatible and will load older style plugins.
The new plugin framework allows plugins to track their dependencies, and
will auto-enable plugins as needed.
Dependencies are tracked via a class-level set named `dependencies` in
each plugin.
Plugin names are no longer tightly coupled with the plugin class name,
Pso EP8 style class names may be used.
Disabling plugins is now allowed, but ensuring proper cleanup is left to
the plugin implementation.
The use of a `post_init()` method is no longer needed for new style
plugins, but plugins following the old style will still require a
`post_init()` method.
Setting self.reconnect_max_attempts to a non-None value will limit
the number of times a connection attempt will be made before quiting
and raising a 'connection_failed' event.
The post_init() system can only reliably handle a single layer
of dependencies between plugins, but PEP plugins with XEP-0115
exceed that limit and plugins can be post_init'ed out of order. To
resolve this, we will special case XEP-0115 to be post_init'ed
first until the new plugin system with dependency tracking is
stable.
Publishes, retractions, purges, and deletions now raise the events:
- pubsub_publish
- pubsub_retract
- pubsub_purge
- pubsub_delete
In addition, custom events may be raised based on the node that
generated the notification. For example:
xmpp['xep_0060'].map_node_event('http://jabber.org/protocol/tune',
'user_tune')
will allow for using the events:
- user_tune_publish
- user_tune_retract
- user_tune_purge
- user_tune_delete
This was XEP-0237, but is now part of RFC 6121.
Roster backends should now expose two additional methods:
version(jid):
Return the version of the given JID's roster.
set_version(jid, version):
Update the version of the given JID's roster.
A new state field will be passed to the backend if an item
has been marked for removal. This is 'removed' which will
be set to True.
Each state element must have its own stanza class now. A stanza class
with an empty name field causes errors in ElementTree, even though
it works fine with cElementTree.
The payload is a dictionary of parsed cert data, as provided by
Python's getpeercert() socket method. It unfortunately does not
provide much detail beyond basic info.
This is inspired by the version from macdiesel and tomstrummer, but
their version was heavily linked with XEP-0096 and focused solely
on file transfer. This version is a more generic implementation.
Added new example for how to retrieve a Google token, following
the best case, non-browser, workflow. Other thirdparty auth
mechs (Facebook, MSN) follow a similar pattern of using an
access token.
This is mainly just useful for authenticating without using TLS.
If an access token is not provided, an attempt will be made to
retrieve one from Google.
Silently substituting the password field was nice, but for mechs
that can use either the password or an access token, it makes
things very difficult. This really only affects MSN clients since
Facebook clients should already be setting the api key.
This requires an extra credential for SASL authentication:
xmpp = ClientXMPP('user@chat.facebook.com', '...access_token...')
xmpp.credentials['api_key'] = '...api_key...'
The parsing and namespace cleaning isn't terribly expensive, but it does
add up. It was adding an extra 5sec when processing 100,000 basic
message stanzas.
Based on profiling, using around 35 stream handlers quarters the number
of basic message stanzas that can be processed in a second, in
comparison to only using the bare minimum of four handlers.
To help, we can drop handlers for stream features once the session
has started. So that we can re-enable these handlers when a stream
must restart, the 'stream_start' event has been added which fires
whenever a stream header is received.
The 'stream_start' event is a more generic replacement for the
existing start_stream_handler() method.
As part of adding this feature:
- fixed bug in update_caps() not assigning verstrings
- fixed xep_0004 typo
- can now use None as a roster key which will map to boundjid.bare
- fixed using JID objects in disco node handlers
- fixed failing test related to get_roster
Several of these bugs I've fixed before, so I either didn't push them
earlier, or I clobbered something when merging. *shrug*
New plugin configuration options:
use_cache - Enable caching disco info results. Defaults to True
wrap_results - Always return disco results in an Iq stanza. Defaults
to False
Node handler changes:
Handlers now take four arguments: jid, node, ifrom, data
Most older style handlers will still work, depending on if they
raise a TypeError for incorrect number of arguments. Handlers that
used *args may not work.
New get_info options:
cached - Passing cached=True to get_info() will attempt to load
results from the cache. If nothing is found, a query
will be sent as normal. If set to False, the cache
will be skipped, even if it contains results.
New method:
supports() - Given a JID/node pair and a feature, return True
if the feature is supported, False if not, and
None if there was a timeout. By default, the search
will use the cache.
Updated the XML-RPC value conversion to correctly apply namespaces, and
fixed an error uncovered by the tests in the XML -> Python conversion of
dateTime values.
This allows applications to filter out sensitive information, such
as passwords, so that it won't appear in the logs.
It does mean that the debug logs will not show the actual received
data, and there will be no indication of tampering, unless the
filter author explicitly logs and notes that a change was made.
A filter accepts and returns a stanza, but potentially modified.
To prevent sending/receiving a stanza, a filter may return None.
Incoming:
self.add_filter('in', in_filter)
Outgoing:
self.add_filter('out', out_filter)
Filters are applied in the order thay are added. However, you may
add an order parameter, which is the place in the list to insert the
filter:
self.add_filter('in', in_filter, order=0)
Only allowing handlers to return a DiscoInfo/DiscoItem stanza works
for the majority of cases, but does not allow for the addition of
an RSM stanza, or other extensions.
An Iq stanza returned by a handler must already be configured as
a reply.
May set self.disconnect_wait=True so that all disconnect
calls wait for the send queue to empty, unless explicitly
overridden with wait=False.
The session_end now fires before closing the socket so
that final stanzas may be sent, such as unavailable presences
for components.
The form plugin was being registered on first use for providers,
but not for clients receiving the form.
NOTE: Use of non-form payloads will have this issue - adhoc command
clients will need to have an expectation beforehand of what
the command payload will be to properly load stanza plugins.
Calling reconnect() simultaneously from multiple threads (like when
using XEP-0199 keepalive) could break because the connection state
can transition and break the state expectations in one of the
reconnect() calls.
Note that using % in a string will _always_ perform the sting substitutions, because the strings are constructed before the function is called. So log.debug('%s' % expensiveoperation()) will take about the same CPU time whether or not the logging level is DEBUG or INFO. if you use , no substitutions are performed unless the string is actually logged
Note that using % in a string will _always_ perform the sting substitutions, because the strings are constructed before the function is called. So log.debug('%s' % expensiveoperation()) will take about the same CPU time whether or not the logging level is DEBUG or INFO. if you use , no substitutions are performed unless the string is actually logged
The change to using the new roster broke the original auto_* values
and used per-roster versions. The original auto_* values will now set
the behaviour globally. Use the per-roster values to override for a
specific JID.
The mechanism name was being correctly de-plussed, but then we used the
original, -PLUS, name to extract the hash, finding SHA-1-PLUS and therefore
finding no match.
Test-Information:
Tested with Sleek against an Isode M-Link with SCRAM-SHA-1-PLUS available.
Author: dwd
Instead of using:
ClientXMPP(jid, password, plugin_config={
'feature_mechanisms': {'use_mech': 'SOME-MECH'}})
You can use:
ClientXMPP(jid, password, sasl_mech='SOME-MECH')
If you need to change the mechanism after instantiation, use:
xmpp['feature_mechanisms'].sasl.mech = 'SCRAM-MD5'
The processing loop was continuing to call __read_xml after </stream>
was received, which caused SyntaxErrors (can't find starting element).
This should fix issue #102
May be disabled by setting:
self.whitespace_keepalive = False
The keepalive interval can be adjusted using:
self.whitespace_keepalive_interval = 300
The default interval is 5min.
Fixes:
README.rst now included
Double line spacing removed from long_description
Source package now includes tests, examples, etc using Manifest.in
README.rst typos fixed
Added README.rst section on installing dnspython for Python3
Version bumped to RC2
Version is now taken from sleekxmpp.version.__version__ without
having to pull in the entire library
Added 'test' command for setup.py
Simplified testall.py
Docs build cleanly from source package after installation
IqError and IqTimeout now extend XMPPError, so if you don't care
about the difference, you can use:
try:
self.do_something_with_iqs()
except XMPPError:
# Error? Timeout? I don't care!
pass
If you do need to distinguish between timeouts and error replies,
you can still continue to use:
try:
self.do_somethin_with_iqs()
except IqError as err:
pass
except IqTimeout:
pass
If you don't catch any Iq errors and you're processing a stanza
then an error response will be sent, just like normal if you raise
XMPPError or any other exception, except that the error messages
will be generic to prevent leaking too much information.
Can now use len(self.client_roster) to get the number of JIDs in
the roster, and self.client_roster.groups() to get a dict of
groups and the JIDs in those groups.
Form fields now remember their current type if the type is deleted. This
allows for fields to properly format their values if set after the form
has been changed to the 'submit' type.
IqError is now caught and forwarded to the command error handler referenced
in the session.
Errors are now caught and processed by the session's error handler
whether or not the results Iq stanza includes the <command> substanza.
Added the option for blocking command calls. The blocking option is set
during start_command with block=True. Subsequent command flow methods use
session['block'] to determine their blocking behaviour.
If you use blocking commands, then you will need to wrap your command calls
in a try/except block for IqTimeout exceptions.
Changes:
May now use underscored method names
form.field is replaced by form['fields']
form.get_fields no longer accepts use_dict parameter, it always
returns an OrderedDict now
form.set_fields will accept either an OrderedDict, or a list
of (var, dict) tuples as before.
Setting the form type to 'submit' will remove extra meta data
from the form fields, leaving just the 'var' and 'value'
Setting the form type to 'cancel' will remove all fields.
Honestly, this is mainly just a demo/proof of concept that we
can handle dependencies and ordering issues with stream features.
DON'T use XEP-0078 if you are able to use the normal SASL method,
which should be the case unless you are dealing with a very old
XMPP server implementation.
Sleek loads a few plugins by default, which made it difficult to
configure or even disable them.
Now, if a plugin is registered without any configuration, then
sleek will try finding a configuration in self.plugin_config.
Thus, using the XEP-0082 and XEP-0202 introduces a dependency
on the dateutil package (installable using pip install python-dateutil).
Maybe we'll be able to rework how these plugins work to avoid
needing dateutil, but for now this will have to do.
Accepting download requests can be done using:
self['xep_0066'].register_url_handler(handler=self.oob_download)
# Add jid=... to specify a handler for a particular JID for a
# componenent.
def oob_download(self, iq):
if iq['from'] not in self.custom_oob_whitelist:
raise XMPPError('not-authorized')
try:
data = urllib2.urlopen(iq['oob_transfer']['url'])
file = open('oob_download', 'w+')
file.write(data.read())
file.close()
data.close()
except:
raise XMPPError('item-not-found')
If wait=True, then the disconnect call will block until
the send queue has emptied.
WARNING: Using wait=True when more stanzas are being added to the
queue than can be processed such that the queue is never empty
will cause the disconnect call to block indefinitely without actually
disconnecting.
The error bubbles through the event processing loop, breaking it and
hanging the application.
Instead, there is now a .exception(e) method on XMLStream which may
be overridden or reassigned that will receive all unhandled exceptions
(read: not XMPPError) from event and stream handlers.
ANONYMOUS was being treated as PLAIN, mechanism was being chosen
purely from supported mechanisms, not those provided by the server.
Broke nested handler methods into top-level methods.
If a stanza handler raised an exception, the exception was processed
and replied by the modified stanza, not a stanza with the original
content.
A copy is now made before handler processing, and if an exception occurs
it is the copy that processes the exception using the original content.
For now, session_end is the same as disconnected, but once support is
added later for stream management, the two events will become distinct.
Plugins should add handlers for session_end for cleaning any session
state.
Backoff was only being done for the initial connection attempt
before. Now any reconnection will start with a minimum 1 sec
delay which will approximately double between attempts.
JIDs with Unicode values were being encoded by the JID class
instead of leaving them as just Unicode strings.
It may still be a good idea to use
from __future__ import unicode_literals
pretty much everywhere though.
Fixes issue #88.
See issue #89
Using get_roster will now return the same types of values as
Iq.send. If a timeout occurs, then the event 'roster_timeout'
will be fired. A successful call to get_roster will also
raise the 'roster_received' event.
To ensure that the get_roster call was successful, here
is a pattern to follow:
def __init__(self, ...):
...
self.add_event_handler('session_start', self.session_start)
self.add_event_handler('roster_timeout', self.roster_timeout)
self.add_event_handler('roster_received', self.roster_received)
def session_start(self, e):
self.send_presence()
self.get_roster()
def roster_timeout(self, e):
# Optionally increase the timeout period
self.get_roster(timeout=self.response_timeout * 2)
def roster_received(self, iq):
# Do stuff, roster has been initialized.
...
Since camelcase names are aliased to the underscored name at startup,
if the underscored version is replaced later, the camelCase name does
not reflect the change.
Each interface, say foo, may be overridden in three ways:
set_foo
get_foo
del_foo
To declare an override in a plugin, add the class field
overrides as so:
overrides = ['set_foo', 'del_foo']
Each override must have a matching set_foo(), etc method
for implementing the new behaviour.
To enable the overrides for a particular parent stanza,
pass the option overrides=True to register_stanza_plugin.
register_stanza_plugin(Stanza, Plugin, overrides=True)
Example code:
class Test(ElementBase):
name = 'test'
namespace = 'testing'
interfaces = set(('foo', 'bar'))
sub_interfaces = set(('bar',))
class TestOverride(ElementBase):
name = 'test-override'
namespace = 'testing'
plugin_attrib = 'override'
interfaces = set(('foo',))
overrides = ['set_foo']
def setup(self, xml):
# Don't include an XML element in the parent stanza
# since we're adding just an attribute.
# If adding a regular subelement, no need to do this.
self.xml = ET.Element('')
def set_foo(self, value):
print("overrides!")
self.parent()._set_attr('foo', 'override-%s' % value)
register_stanza_plugin(Test, TestOverride, overrides=True)
Example usage:
>>> t = TestStanza()
>>> t['foo'] = 'bar'
>>> t['foo']
'override-bar'
Backward incompatibility alert!
Please see examples/adhoc_provider.py for how to use the new
plugin implementation, or the test examples in the files
tests/test_stream_xep_0050.py and tests/test_stanza_xep_0050.py.
Major changes:
- May now have zero-step commands. Useful if a command is
intended to be a dynamic status report that doesn't
require any user input.
- May use payloads other than data forms, such as a
completely custom stanza type.
- May include multiple payload items, such as multiple
data forms, or a form and a custom stanza type.
- Includes a command user API for calling adhoc commands
on remote agents and managing the workflow.
- Added support for note elements.
Todo:
- Add prev action support.
You may use register_plugin('old_0050') to continue using the
previous XEP-0050 implementation.
ElementBase instances will display the top-most namespace by default.
StanzaBase instances will NOT display the top-most namespace by default.
May pass True or False to __str__ to override.
We need to check if type="get". otherwise we will send our version
when we will receive the version of the remote entity, and thus
going in an infinite loop.
Can now be used as so:
>>> msg['chat_state']
''
>>> msg
<message />
>>> msg['chat_state'] = 'paused'
>>> msg
<message>
<paused xmlns="http://jabber.org/protocol/chatstates" />
</message>
>>> msg['chat_state']
'paused'
>>> del msg['chat_state']
>>> msg
<message />
Now done more responsibly, saving any existing signal handlers
and calling them when an interrupt occurs in addition to the
one Sleek installs.
NOTE: You may need to explicitly use "kill <process id>" in
order to trigger the proper signal handler execution, and
to raise the "killed" event.
Instead of the actual callback object, return just the name of
the callback object created when using iq.send(callback=..).
This will help prevent memory leaks by not keeping an additional
reference to the object, but still allows for the callback to be
canceled by using self.remove_handler("handler_name").
Waiting until the actual run step means that the handler is not
marked for deletion when checked in the __spawn_event() thread,
causing the callback to stay in the handler list.
This allows exceptions to include the original
content of a stanza in the error response by including
the parameter clear=False when raising the exception.
Stream features now use stanza objects!
Features are given a ranking that expresses the dependency
relationships (since only one feature is negotiated at a time, the
dependency graph can be replaced by a line).
>>> xmpp.register_feature('my_feature', _my_handler,
>>> restart=True, # Requires stream restart
>>> order=600) # Ranking (out of ~ 10,000,
>>> # lower #'s executed first)
SASL mechanisms may now be added or disabled as needed. Each mechanism
is given a priority value indicating the order in which the client
wishes for mechanisms to be tried. Higher priority numbers are executed
first.
>>> xmpp.register_sasl_mechanism('SASL-MECH', _mech_handler,
>>> priority=0)
Disabling a SASL mechanism:
>>> xmpp.remove_sasl_mechanism('ANONYMOUS')
Will now always show top-level namespace, unless it is the same
as the stream's default namespace. Also added the XMPP stream
namespace to the namespace map as 'stream'.
Use stanza.values instead of _get/set_stanza_values where used.
ElementBase stanzas can now use .tag
May use class method tag_name() for stanza classes.
ElementBase now has .clear() method.
May now use register_stanza_plugin(Foo, Bar, iterable=True)
to add to the set of stanza classes used for iterable
substanzas. It is no longer necessary to manually specify
the contents of subitem if the new method is used.
A stanza object may add is_extension = True to its class definition
to provide a single new interface to a parent stanza.
For example:
import sleekxmpp
from sleekxmpp import Iq
from sleekxmpp.xmlstream import ElementBase, register_stanza_plugin, ET
class Foo(ElementBase):
"""
Test adding just an attribute to a parent stanza.
Adding subelements works as expected.
"""
is_extension = True
interfaces = set(('foo',))
plugin_attrib = 'foo'
def setup(self, xml):
# Don't include an XML element in the parent stanza
# since we're adding just an attribute.
# If adding a regular subelement, no need to do this.
self.xml = ET.Element('')
def set_foo(self, val):
self.parent()._set_attr('foo', val)
def get_foo(self):
return self.parent()._get_attr('foo')
def del_foo(self):
self.parent()._del_attr('foo')
register_stanza_plugin(Iq, Foo)
i1 = Iq()
i2 = Iq(xml=ET.fromstring("<iq xmlns='jabber:client' foo='bar' />"))
>>> i1['foo'] = '3'
>>> i1
'3'
>>> i1
'<iq id="0" foo="3" />'
>>> i2
'<iq id="0" foo="bar" />'
>>> i2['foo']
'bar'
>>> del i2['foo']
>>> i2
'<iq id="0" />'
We now raise the unexpected exceptions instead of sending
them on the network.
- avoids flood (sending a traceback on a MUC, for example…) and
maybe some security issues.
- lets you handle the traceback (catch it to handle
it properly, or with except_hook, etc)
- an exception cannot be raised without you knowing
Daemonized threads exit once the main program has quit,
and the only threads left running are all daemon threads.
Should fix hanging clients while not trampling over anyone
else's signal handlers.
Methods now accept either an ifrom or mfrom parameter
to specify a 'from' value. Client connections should not
need to use these, but component connections must use them.
May include a to and from JID in make_iq_* calls.
May pass an existing iq stanza to most of them instead of generating
a new stanza.
make_iq now accepts a 'to' value, 'type' value, and 'query' value to
simplify things a bit more.
Now with dynamic node handling goodness.
Some things are not quite working yet, in particular:
set_items
set_info
set_identities
set_features
And still need more unit tests to round things out.
Support is only for adding literal XML content
to stanzas. Full support for things like multiple
message bodies with different xml:lang values is
still in the works.
The callback will be a stream level handler, and will not
execute in its own thread. If you must have a thread, have the
callback function raise a custom event, which can be processed
by another event handler, which may run in an individual thread,
like so:
def handle_reply(self, iq):
self.event('custom_event', iq)
def do_long_operation_in_thread(self, iq):
...
self.add_event_handler('custom_event', self.do_long_operation_in_thread)
...take out already prepared iq stanza...
iq.send(callback=self.handle_reply)
SleekTest can now use matchers when checking stanzas, using
the method parameter for self.check(), self.recv(), and self.send():
method='exact' - Same behavior as before
'xpath' - Use xpath matcher
'id' - Use ID matcher
'mask' - Use XML mask matcher
'stanzapath' - Use StanzaPath matcher
recv_feature and send_feature only accept 'exact' and 'mask' for now.
