Sieve stopped working after updating cyrus-imapd to 3.0.5

Discussion:

rfk

2018-02-14 18:28:45 UTC

Hello,

recently I updated my cyrus-imapd installation from verion 2.5.10 to
version 3.0.5. Unfortunately the sieve filters stopped working since
then. The rest is working fine.

The filters are listed in the sieveshell and all incoming mails get
tagged with "X-Sieve: CMU Sieve 3.0". However even the simplest rules
are just getting ignored. In the logs I cannot find any related entries.

My configuration settings regarding "unixhierarchysep" (on) and
"altnamespace" (off) have not changed since version 2.5.10 (I configured
them accordingly in version 3.0.5). I followed the entire upgrade guide
by the way.

Furthermore I also tried adding a completely new account with a very
simple new sieve script (mark as seen and move to folder; created by
Roundcube) but nothing happens to incoming mails.

Is there anything else I can check? Is it possible to raise the log
level for sieve?

Thanks in advance.

Ralf

ellie timoney

2018-02-15 02:39:34 UTC

Permalink

Hi,

Does the "sievec" compiler report any errors if you try to compile your sieve scripts by hand?

Its manpage is here: https://www.cyrusimap.org/imap/reference/manpages/systemcommands/sievec.html

Cheers,

ellie

Post by rfk
Hello,
recently I updated my cyrus-imapd installation from verion 2.5.10 to
version 3.0.5. Unfortunately the sieve filters stopped working since
then. The rest is working fine.
The filters are listed in the sieveshell and all incoming mails get
tagged with "X-Sieve: CMU Sieve 3.0". However even the simplest rules
are just getting ignored. In the logs I cannot find any related entries.
My configuration settings regarding "unixhierarchysep" (on) and
"altnamespace" (off) have not changed since version 2.5.10 (I configured
them accordingly in version 3.0.5). I followed the entire upgrade guide
by the way.
Furthermore I also tried adding a completely new account with a very
simple new sieve script (mark as seen and move to folder; created by
Roundcube) but nothing happens to incoming mails.
Is there anything else I can check? Is it possible to raise the log
level for sieve?
Thanks in advance.
Ralf
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

rfk

2018-02-15 17:06:23 UTC

Permalink

Hi,

thanks for your feedback. Unfortunately sievec reports nothing:

# grep -v ^# roundcube.script
require ["imapflags","fileinto"];

if anyof (header :contains "Subject" "test")
{
setflag "\\Seen";
fileinto "INBOX/Trash";
stop;
}

# sievec roundcube.script test
#

Here's my configuration in case it helps:

# grep -vE "^#|^$" /etc/imapd.conf
admins: cyrus
configdirectory: /var/imap
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
duplicate_db_path: /run/cyrus/deliver.db
ptscache_db_path: /run/cyrus/ptscache.db
statuscache_db_path: /run/cyrus/statuscache.db
tls_sessions_db_path: /run/cyrus/tls_sessions.db
defaultpartition: default
partition-default: /var/spool/imap
sievedir: /var/imap/sieve
lmtpsocket: /var/imap/socket/lmtp
idlesocket: /var/imap/socket/idle
notifysocket: /var/imap/socket/notify
syslog_prefix: cyrus
httpmodules: caldav carddav
hashimapspool: true
virtdomains: no
defaultdomain: localhost
popminpoll: 1
allowplaintext: yes
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
sasl_auto_transition: no
sieve_extensions: fileinto reject vacation imapflags notify include
envelope body relational regex subaddress copy
tls_session_timeout: 1440
tls_client_ca_file: /.../ca_file.pem
tls_server_cert: /.../cert.pem
tls_server_key: /.../key.pem

# grep -vE "^\s*#|^$" /etc/cyrus.conf
START {
recover cmd="ctl_cyrusdb -r"
}
SERVICES {
imap cmd="imapd" listen="imap" prefork=0
imaps cmd="imapd -s" listen="imaps" prefork=0
sieve cmd="timsieved" listen="sieve" prefork=0
lmtpunix cmd="lmtpd" listen="/var/imap/socket/lmtp" prefork=0
}
EVENTS {
checkpoint cmd="ctl_cyrusdb -c" period=30
delprune cmd="cyr_expire -E 3" at=0400
deleteprune cmd="cyr_expire -E 4 -D 38" at=0430
expungeprune cmd="cyr_expire -E 4 -X 38" at=0445
tlsprune cmd="tls_prune" at=0400
squatter cmd="squatter -r *" period=30
}
DAEMON {
idled cmd="idled"
}

Any other ideas?

Ralf

Post by ellie timoney
Hi,
Does the "sievec" compiler report any errors if you try to compile your sieve scripts by hand?
Its manpage is here: https://www.cyrusimap.org/imap/reference/manpages/systemcommands/sievec.html
Cheers,
ellie

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Michael Menge

2018-02-15 18:32:42 UTC

Permalink

Hi,

Post by rfk
Hi,
# grep -v ^# roundcube.script
require ["imapflags","fileinto"];
if anyof (header :contains "Subject" "test")
{
setflag "\\Seen";
fileinto "INBOX/Trash";
stop;
}
# sievec roundcube.script test
#
# grep -vE "^#|^$" /etc/imapd.conf
admins: cyrus
configdirectory: /var/imap
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
duplicate_db_path: /run/cyrus/deliver.db
ptscache_db_path: /run/cyrus/ptscache.db
statuscache_db_path: /run/cyrus/statuscache.db
tls_sessions_db_path: /run/cyrus/tls_sessions.db
defaultpartition: default
partition-default: /var/spool/imap
sievedir: /var/imap/sieve
lmtpsocket: /var/imap/socket/lmtp
idlesocket: /var/imap/socket/idle
notifysocket: /var/imap/socket/notify
syslog_prefix: cyrus
httpmodules: caldav carddav
hashimapspool: true
virtdomains: no
defaultdomain: localhost
popminpoll: 1
allowplaintext: yes
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
sasl_auto_transition: no
sieve_extensions: fileinto reject vacation imapflags notify include
envelope body relational regex subaddress copy
tls_session_timeout: 1440
tls_client_ca_file: /.../ca_file.pem
tls_server_cert: /.../cert.pem
tls_server_key: /.../key.pem
Any other ideas?

the altnamespace default cahnged in 3.0. so you should disable it in
imapd.conf or change the fileinto to "Trash"

Quoting imapd.conf manpage
====================
altnamespace: 1

Use the alternate IMAP namespace, where personal folders reside at
the same level in the hierarchy as INBOX.
This option ONLY applies where interaction takes place with the
client/user. Currently this is limited to
the IMAP protocol (imapd) and Sieve scripts (lmtpd). This option
does NOT apply to admin tools such as
cyradm (admins ONLY), reconstruct, quota, etc., NOR does it affect
LMTP delivery of messages directly to
mailboxes via plus-addressing. The default changed in 3.0 from off to on.
===================

Post by rfk
Ralf

Post by ellie timoney
Hi,
Does the "sievec" compiler report any errors if you try to compile
your sieve scripts by hand?
https://www.cyrusimap.org/imap/reference/manpages/systemcommands/sievec.html
Cheers,
ellie

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

--------------------------------------------------------------------------------
M.Menge Tel.: (49) 7071/29-70316
Universität Tübingen Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung mail:
***@zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https

rfk

2018-02-15 18:37:19 UTC

Permalink

Hi,

I'm so sorry, don't know how this could happen, but the last line of my
configuration was lost during copy and paste:

altnamespace: 0

So it is already disabled.

Ralf

Post by Michael Menge
Hi,

Post by rfk
Hi,
# grep -v ^# roundcube.script
require ["imapflags","fileinto"];
if anyof (header :contains "Subject" "test")
{
        setflag "\\Seen";
        fileinto "INBOX/Trash";
        stop;
}
# sievec roundcube.script test
#
# grep -vE "^#|^$" /etc/imapd.conf
admins: cyrus
configdirectory: /var/imap
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
duplicate_db_path: /run/cyrus/deliver.db
ptscache_db_path: /run/cyrus/ptscache.db
statuscache_db_path: /run/cyrus/statuscache.db
tls_sessions_db_path: /run/cyrus/tls_sessions.db
defaultpartition: default
partition-default: /var/spool/imap
sievedir: /var/imap/sieve
lmtpsocket: /var/imap/socket/lmtp
idlesocket: /var/imap/socket/idle
notifysocket: /var/imap/socket/notify
syslog_prefix: cyrus
httpmodules: caldav carddav
hashimapspool: true
virtdomains: no
defaultdomain: localhost
popminpoll: 1
allowplaintext: yes
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
sasl_auto_transition: no
sieve_extensions: fileinto reject vacation imapflags notify include
envelope body relational regex subaddress copy
tls_session_timeout: 1440
tls_client_ca_file:     /.../ca_file.pem
tls_server_cert:        /.../cert.pem
tls_server_key:         /.../key.pem
Any other ideas?

the altnamespace default cahnged in 3.0. so you should disable it in
imapd.conf or change the fileinto to "Trash"
Quoting imapd.conf manpage
====================
altnamespace: 1
Use the alternate IMAP namespace, where personal folders reside at
the same level in the hierarchy as INBOX.
This option ONLY applies where interaction takes place with the
client/user. Currently this is limited to
the IMAP protocol (imapd) and Sieve scripts (lmtpd). This option does
NOT apply to admin tools such as
cyradm (admins ONLY), reconstruct, quota, etc., NOR does it affect
LMTP delivery of messages directly to
mailboxes via plus-addressing. The default changed in 3.0 from off to
on.
===================

Post by rfk
Ralf

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

--------------------------------------------------------------------------------
M.Menge Tel.: (49) 7071/29-70316
Universität Tübingen Fax.: (49) 7071/29-5912
Wächterstraße 76
72074 Tübingen
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andre

rfk

2018-02-28 18:18:55 UTC

Permalink

Hello,

still no other ideas?

Ralf

Post by rfk
Hi,
I'm so sorry, don't know how this could happen, but the last line of my
altnamespace: 0
So it is already disabled.
Ralf

Post by Michael Menge
Hi,

Post by rfk
Hi,
# grep -v ^# roundcube.script
require ["imapflags","fileinto"];
if anyof (header :contains "Subject" "test")
{
        setflag "\\Seen";
        fileinto "INBOX/Trash";
        stop;
}
# sievec roundcube.script test
#
# grep -vE "^#|^$" /etc/imapd.conf
admins: cyrus
configdirectory: /var/imap
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
duplicate_db_path: /run/cyrus/deliver.db
ptscache_db_path: /run/cyrus/ptscache.db
statuscache_db_path: /run/cyrus/statuscache.db
tls_sessions_db_path: /run/cyrus/tls_sessions.db
defaultpartition: default
partition-default: /var/spool/imap
sievedir: /var/imap/sieve
lmtpsocket: /var/imap/socket/lmtp
idlesocket: /var/imap/socket/idle
notifysocket: /var/imap/socket/notify
syslog_prefix: cyrus
httpmodules: caldav carddav
hashimapspool: true
virtdomains: no
defaultdomain: localhost
popminpoll: 1
allowplaintext: yes
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
sasl_auto_transition: no
sieve_extensions: fileinto reject vacation imapflags notify include
envelope body relational regex subaddress copy
tls_session_timeout: 1440
tls_client_ca_file:     /.../ca_file.pem
tls_server_cert:        /.../cert.pem
tls_server_key:         /.../key.pem
Any other ideas?

the altnamespace default cahnged in 3.0. so you should disable it in
imapd.conf or change the fileinto to "Trash"
Quoting imapd.conf manpage
====================
altnamespace: 1
   Use the alternate IMAP namespace, where personal folders reside at
the same level in the hierarchy as INBOX.
   This option ONLY applies where interaction takes place with the
client/user. Currently this is limited to
   the IMAP protocol (imapd) and Sieve scripts (lmtpd). This option
does NOT apply to admin tools such as
   cyradm (admins ONLY), reconstruct, quota, etc., NOR does it affect
LMTP delivery of messages directly to
   mailboxes via plus-addressing. The default changed in 3.0 from off
to on.
===================

Post by rfk
Ralf

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinf

Sebastian Hagedorn

2018-03-01 12:52:15 UTC

Permalink

This post might be inappropriate. Click to display it.

rfk

2018-03-01 20:36:47 UTC

Permalink

Hey Sebastian,

thanks a lot for your feedback, seems like we're coming closer. Although
I'm not sure what port 24 is supposed to be in your swaks command I just
sent myself an e-mail while strace was attached to cyrusmaster.

The log revealed the following:
readlink("/var/imap/sieve/u/user^domain^de/defaultbc", 0x7fff7fa31d00,
4096) = -1 ENOENT (No such file or directory)

Then I recognized that the users in /var/imap/sieve/ are actually
separated by a dot ('.') instead of a caret ('^') as opposed to
/var/spool/imap/ where the users are separated by a caret. After
manually renaming the sieve directory of my user to the caret notation
the filtering was finally working.

However Roundcube then tells me that no filters are available. Once I
used Roundcube (or sieveshell) my sieve user directory is existing
twice, once empty in dot notation and once with the filters in caret
notation.

My conclusion:
1. lmtpd (?) is using the caret notation when looking up sieve scripts.
2. timsieved (?) is using the dot notation when creating sieve scripts.

Which configuration parameters affect this behaviour?

Regards,
Ralf

Post by Sebastian Hagedorn
Not really. My best suggestion would be to strace the lmtp process to
see if the sieve scripts are even found and opened. On a test system I
strace -f -ff -o /tmp/lmtptrace -p <PID of cyrus-master>
It might be better to prefork an lmtpd and to specifically strace that
pid, but in my case it didn't matter.
swaks --protocol lmtp -t name-of-a-user-with-sieve -s localhost -p 24
lmtptrace.43344:open("/lib64/libcyrus_sieve.so.0", O_RDONLY|O_CLOEXEC) = 5
lmtptrace.43344:readlink("/var/lib/imap/sieve/E/xxx/defaultbc",
"smartsieve.bc", 4096) = 13
lmtptrace.43344:open("/var/lib/imap/sieve/E/xxx/smartsieve.script",
O_RDWR) = 12
lmtptrace.43344:stat("/var/lib/imap/sieve/E/xxx/smartsieve.bc",
{st_mode=S_IFREG|0600, st_size=604, ...}) = 0
lmtptrace.43344:stat("/var/lib/imap/sieve/E/xxx/smartsieve.bc",
{st_mode=S_IFREG|0600, st_size=604, ...}) = 0
lmtptrace.43344:open("/var/lib/imap/sieve/E/xxx/smartsieve.bc",
O_RDONLY) = 13
lmtptrace.43344:stat("/var/lib/imap/sieve/E/xxx/defaultbc",
{st_mode=S_IFREG|0600, st_size=604, ...}) = 0
lmtptrace.43344:open("/var/lib/imap/sieve/E/xxx/defaultbc", O_RDONLY) = 12
Maybe try that and report back what happens on your system.

Post by rfk
Hello,
still no other ideas?
Ralf

Post by rfk
Hi,
I'm so sorry, don't know how this could happen, but the last line of my
altnamespace: 0
So it is already disabled.
Ralf

Post by Michael Menge
Hi,

Post by rfk
Hi,
# grep -v ^# roundcube.script
require ["imapflags","fileinto"];
if anyof (header :contains "Subject" "test")
{
        setflag "\\Seen";
        fileinto "INBOX/Trash";
        stop;
}
# sievec roundcube.script test
#
# grep -vE "^#|^$" /etc/imapd.conf
admins: cyrus
configdirectory: /var/imap
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
duplicate_db_path: /run/cyrus/deliver.db
ptscache_db_path: /run/cyrus/ptscache.db
statuscache_db_path: /run/cyrus/statuscache.db
tls_sessions_db_path: /run/cyrus/tls_sessions.db
defaultpartition: default
partition-default: /var/spool/imap
sievedir: /var/imap/sieve
lmtpsocket: /var/imap/socket/lmtp
idlesocket: /var/imap/socket/idle
notifysocket: /var/imap/socket/notify
syslog_prefix: cyrus
httpmodules: caldav carddav
hashimapspool: true
virtdomains: no
defaultdomain: localhost
popminpoll: 1
allowplaintext: yes
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
sasl_auto_transition: no
sieve_extensions: fileinto reject vacation imapflags notify include
envelope body relational regex subaddress copy
tls_session_timeout: 1440
tls_client_ca_file:     /.../ca_file.pem
tls_server_cert:        /.../cert.pem
tls_server_key:         /.../key.pem
Any other ideas?

the altnamespace default cahnged in 3.0. so you should disable it in
imapd.conf or change the fileinto to "Trash"
Quoting imapd.conf manpage
====================
altnamespace: 1
   Use the alternate IMAP namespace, where personal folders reside at
the same level in the hierarchy as INBOX.
   This option ONLY applies where interaction takes place with the
client/user. Currently this is limited to
   the IMAP protocol (imapd) and Sieve scripts (lmtpd). This option
does NOT apply to admin tools such as
   cyradm (admins ONLY), reconstruct, quota, etc., NOR does it affect
LMTP delivery of messages directly to
   mailboxes via plus-addressing. The default changed in 3.0 from off
to on.
===================

Post by rfk
Ralf

Post by rfk
Hello,
recently I updated my cyrus-imapd installation from verion 2.5.10 to
version 3.0.5. Unfortunately the sieve filters stopped working since
then. The rest is working fine.
The filters are listed in the sieveshell and all incoming mails get
tagged with "X-Sieve: CMU Sieve 3.0". However even the simplest rules
are just getting ignored. In the logs I cannot find any related entries.
My configuration settings regarding "unixhierarchysep" (on) and
"altnamespace" (off) have not changed since version 2.5.10 (I configured
them accordingly in version 3.0.5). I followed the entire upgrade guide
by the way.
Furthermore I also tried adding a completely new account with a very
simple new sieve script (mark as seen and move to folder; created by
Roundcube) but nothing happens to incoming mails.
Is there anything else I can check? Is it possible to raise the log
level for sieve?
Thanks in advance.
Ralf
----
Cyrus Home Page: http://www.cyrusimap.org/
http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-----------------------------------------------------------------------
---------
M.Menge
Tel.: (49) 7071/29-70316 Universität
Tübingen Fax.: (49) 7071/29-5912
Wächterstraße 76
72074 Tübingen
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Mit freundlichen Grüßen
Sebastian Hagedorn

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/lis

Sebastian Hagedorn

2018-03-02 08:21:25 UTC

Permalink

Hi Ralf,

Post by rfk
thanks a lot for your feedback, seems like we're coming closer. Although
I'm not sure what port 24 is supposed to be in your swaks command I just
sent myself an e-mail while strace was attached to cyrusmaster.
readlink("/var/imap/sieve/u/user^domain^de/defaultbc", 0x7fff7fa31d00,
4096) = -1 ENOENT (No such file or directory)
Then I recognized that the users in /var/imap/sieve/ are actually
separated by a dot ('.') instead of a caret ('^') as opposed to
/var/spool/imap/ where the users are separated by a caret. After manually
renaming the sieve directory of my user to the caret notation the
filtering was finally working.
However Roundcube then tells me that no filters are available. Once I
used Roundcube (or sieveshell) my sieve user directory is existing twice,
once empty in dot notation and once with the filters in caret notation.
1. lmtpd (?) is using the caret notation when looking up sieve scripts.
2. timsieved (?) is using the dot notation when creating sieve scripts.
Which configuration parameters affect this behaviour?

the only explanation I can come up with is that you have more than one
imapd.conf and that the two daemons use different ones. Do any of the
entries in your cyrus.conf use the "-C" flag? I'm not an expert on that,
but I seem to remember that the caret notation is part of virtual domain
support. I have no experience with that because we don't need it.
--
Sebastian Hagedorn - Weyertal 121, Zimmer 2.02
Regionales Rechenzentrum (RRZK)
UniversitÃ€t zu KÃ¶ln / Cologne University - Tel. +49-221-470-89578

rfk

2018-03-02 19:03:07 UTC

Permalink

Hi Sebastian,

in my second post you can see my cyrus.conf:
https://lists.andrew.cmu.edu/pipermail/info-cyrus/2018-February/039985.html

No "-C" flags in use though.

Regards,
Ralf

Post by Sebastian Hagedorn
Hi Ralf,

the only explanation I can come up with is that you have more than one
imapd.conf and that the two daemons use different ones. Do any of the
entries in your cyrus.conf use the "-C" flag? I'm not an expert on that,
but I seem to remember that the caret notation is part of virtual domain
support. I have no experience with that because we don't need it.
--
Sebastian Hagedorn - Weyertal 121, Zimmer 2.02
Regionales Rechenzentrum (RRZK)
Universität zu Köln / Cologne University - Tel. +49-221-470-89578

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/lis

Ken Murchison

2018-03-02 15:50:45 UTC

Permalink

I've done some investigating this morning and it appears that you are
correct in that lmtpd and timsieved differ in how they handle the
hierarchy separator. But I'm not sure just yet when this split happened.

Did you upgrade from 3.0.4 or from 2.x?

Post by rfk
Hey Sebastian,
thanks a lot for your feedback, seems like we're coming closer.
Although I'm not sure what port 24 is supposed to be in your swaks
command I just sent myself an e-mail while strace was attached to
cyrusmaster.
readlink("/var/imap/sieve/u/user^domain^de/defaultbc", 0x7fff7fa31d00,
4096) = -1 ENOENT (No such file or directory)
Then I recognized that the users in /var/imap/sieve/ are actually
separated by a dot ('.') instead of a caret ('^') as opposed to
/var/spool/imap/ where the users are separated by a caret. After
manually renaming the sieve directory of my user to the caret notation
the filtering was finally working.
However Roundcube then tells me that no filters are available. Once I
used Roundcube (or sieveshell) my sieve user directory is existing
twice, once empty in dot notation and once with the filters in caret
notation.
1. lmtpd (?) is using the caret notation when looking up sieve scripts.
2. timsieved (?) is using the dot notation when creating sieve scripts.
Which configuration parameters affect this behaviour?
Regards,
Ralf

Post by Sebastian Hagedorn
Not really. My best suggestion would be to strace the lmtp process to
see if the sieve scripts are even found and opened. On a test system
strace -f -ff -o /tmp/lmtptrace -p <PID of cyrus-master>
It might be better to prefork an lmtpd and to specifically strace
that pid, but in my case it didn't matter.
swaks --protocol lmtp -t name-of-a-user-with-sieve -s localhost -p 24
lmtptrace.43344:open("/lib64/libcyrus_sieve.so.0",
O_RDONLY|O_CLOEXEC) = 5
lmtptrace.43344:readlink("/var/lib/imap/sieve/E/xxx/defaultbc",
"smartsieve.bc", 4096) = 13
lmtptrace.43344:open("/var/lib/imap/sieve/E/xxx/smartsieve.script",
O_RDWR) = 12
lmtptrace.43344:stat("/var/lib/imap/sieve/E/xxx/smartsieve.bc",
{st_mode=S_IFREG|0600, st_size=604, ...}) = 0
lmtptrace.43344:stat("/var/lib/imap/sieve/E/xxx/smartsieve.bc",
{st_mode=S_IFREG|0600, st_size=604, ...}) = 0
lmtptrace.43344:open("/var/lib/imap/sieve/E/xxx/smartsieve.bc",
O_RDONLY) = 13
lmtptrace.43344:stat("/var/lib/imap/sieve/E/xxx/defaultbc",
{st_mode=S_IFREG|0600, st_size=604, ...}) = 0
lmtptrace.43344:open("/var/lib/imap/sieve/E/xxx/defaultbc", O_RDONLY) = 12
Maybe try that and report back what happens on your system.

Post by rfk
Hello,
still no other ideas?
Ralf

Post by rfk
Hi,
I'm so sorry, don't know how this could happen, but the last line of my
altnamespace: 0
So it is already disabled.
Ralf

Post by Michael Menge
Hi,

Post by rfk
Hi,
# grep -v ^# roundcube.script
require ["imapflags","fileinto"];
if anyof (header :contains "Subject" "test")
{
        setflag "\\Seen";
        fileinto "INBOX/Trash";
        stop;
}
# sievec roundcube.script test
#
# grep -vE "^#|^$" /etc/imapd.conf
admins: cyrus
configdirectory: /var/imap
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
duplicate_db_path: /run/cyrus/deliver.db
ptscache_db_path: /run/cyrus/ptscache.db
statuscache_db_path: /run/cyrus/statuscache.db
tls_sessions_db_path: /run/cyrus/tls_sessions.db
defaultpartition: default
partition-default: /var/spool/imap
sievedir: /var/imap/sieve
lmtpsocket: /var/imap/socket/lmtp
idlesocket: /var/imap/socket/idle
notifysocket: /var/imap/socket/notify
syslog_prefix: cyrus
httpmodules: caldav carddav
hashimapspool: true
virtdomains: no
defaultdomain: localhost
popminpoll: 1
allowplaintext: yes
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
sasl_auto_transition: no
sieve_extensions: fileinto reject vacation imapflags notify include
envelope body relational regex subaddress copy
tls_session_timeout: 1440
tls_client_ca_file:     /.../ca_file.pem
tls_server_cert:        /.../cert.pem
tls_server_key:         /.../key.pem
Any other ideas?

the altnamespace default cahnged in 3.0. so you should disable it in
imapd.conf or change the fileinto to "Trash"
Quoting imapd.conf manpage
====================
altnamespace: 1
   Use the alternate IMAP namespace, where personal folders reside at
the same level in the hierarchy as INBOX.
   This option ONLY applies where interaction takes place with the
client/user. Currently this is limited to
   the IMAP protocol (imapd) and Sieve scripts (lmtpd). This option
does NOT apply to admin tools such as
   cyradm (admins ONLY), reconstruct, quota, etc., NOR does it affect
LMTP delivery of messages directly to
   mailboxes via plus-addressing. The default changed in 3.0 from off
to on.
===================

Post by rfk
Ralf

Post by rfk
Hello,
recently I updated my cyrus-imapd installation from verion 2.5.10 to
version 3.0.5. Unfortunately the sieve filters stopped working since
then. The rest is working fine.
The filters are listed in the sieveshell and all incoming mails get
tagged with "X-Sieve: CMU Sieve 3.0". However even the simplest rules
are just getting ignored. In the logs I cannot find any related entries.
My configuration settings regarding "unixhierarchysep" (on) and
"altnamespace" (off) have not changed since version 2.5.10 (I configured
them accordingly in version 3.0.5). I followed the entire upgrade guide
by the way.
Furthermore I also tried adding a completely new account with a very
simple new sieve script (mark as seen and move to folder; created by
Roundcube) but nothing happens to incoming mails.
Is there anything else I can check? Is it possible to raise the log
level for sieve?
Thanks in advance.
Ralf
----
Cyrus Home Page: http://www.cyrusimap.org/
http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-----------------------------------------------------------------------
---------
M.Menge
Tel.: (49) 7071/29-70316 Universität
Tübingen Fax.: (49) 7071/29-5912
Wächterstraße 76
72074 Tübingen
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Mit freundlichen Grüßen
Sebastian Hagedorn

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

--
Kenneth Murchison
Cyrus Development Team
FastMail Pty Ltd

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://list

rfk

2018-03-02 19:11:59 UTC

Permalink

Hi Ken,

my distribution is Gentoo and my previous cyrus version was 2.5.10. Then
I upgraded to 3.0.4 but this ebuild (though marked stable by the Gentoo
devs) was terminating "abnormally" all the time. Finally I upgraded to
3.0.5 which is running fine until now - well, except for sieve.

Regards,
Ralf

Post by Ken Murchison
I've done some investigating this morning and it appears that you are
correct in that lmtpd and timsieved differ in how they handle the
hierarchy separator. But I'm not sure just yet when this split happened.
Did you upgrade from 3.0.4 or from 2.x?

Post by rfk
Hello,
still no other ideas?
Ralf

Post by rfk
Hi,
I'm so sorry, don't know how this could happen, but the last line of my
altnamespace: 0
So it is already disabled.
Ralf

Post by Michael Menge
Hi,

Post by rfk
Hi,
# grep -v ^# roundcube.script
require ["imapflags","fileinto"];
if anyof (header :contains "Subject" "test")
{
        setflag "\\Seen";
        fileinto "INBOX/Trash";
        stop;
}
# sievec roundcube.script test
#
# grep -vE "^#|^$" /etc/imapd.conf
admins: cyrus
configdirectory: /var/imap
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
duplicate_db_path: /run/cyrus/deliver.db
ptscache_db_path: /run/cyrus/ptscache.db
statuscache_db_path: /run/cyrus/statuscache.db
tls_sessions_db_path: /run/cyrus/tls_sessions.db
defaultpartition: default
partition-default: /var/spool/imap
sievedir: /var/imap/sieve
lmtpsocket: /var/imap/socket/lmtp
idlesocket: /var/imap/socket/idle
notifysocket: /var/imap/socket/notify
syslog_prefix: cyrus
httpmodules: caldav carddav
hashimapspool: true
virtdomains: no
defaultdomain: localhost
popminpoll: 1
allowplaintext: yes
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
sasl_auto_transition: no
sieve_extensions: fileinto reject vacation imapflags notify include
envelope body relational regex subaddress copy
tls_session_timeout: 1440
tls_client_ca_file:     /.../ca_file.pem
tls_server_cert:        /.../cert.pem
tls_server_key:         /.../key.pem
Any other ideas?

the altnamespace default cahnged in 3.0. so you should disable it in
imapd.conf or change the fileinto to "Trash"
Quoting imapd.conf manpage
====================
altnamespace: 1
   Use the alternate IMAP namespace, where personal folders reside at
the same level in the hierarchy as INBOX.
   This option ONLY applies where interaction takes place with the
client/user. Currently this is limited to
   the IMAP protocol (imapd) and Sieve scripts (lmtpd). This option
does NOT apply to admin tools such as
   cyradm (admins ONLY), reconstruct, quota, etc., NOR does it affect
LMTP delivery of messages directly to
   mailboxes via plus-addressing. The default changed in 3.0 from off
to on.
===================

Post by rfk
Ralf

Post by rfk
Hello,
recently I updated my cyrus-imapd installation from verion 2.5.10 to
version 3.0.5. Unfortunately the sieve filters stopped working since
then. The rest is working fine.
The filters are listed in the sieveshell and all incoming mails get
tagged with "X-Sieve: CMU Sieve 3.0". However even the simplest rules
are just getting ignored. In the logs I cannot find any related entries.
My configuration settings regarding "unixhierarchysep" (on) and
"altnamespace" (off) have not changed since version 2.5.10 (I configured
them accordingly in version 3.0.5). I followed the entire upgrade guide
by the way.
Furthermore I also tried adding a completely new account with a very
simple new sieve script (mark as seen and move to folder; created by
Roundcube) but nothing happens to incoming mails.
Is there anything else I can check? Is it possible to raise the log
level for sieve?
Thanks in advance.
Ralf
----
Cyrus Home Page: http://www.cyrusimap.org/
http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-----------------------------------------------------------------------
---------
M.Menge
Tel.: (49) 7071/29-70316 Universität
Tübingen Fax.: (49) 7071/29-5912
Wächterstraße 76
72074 Tübingen
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Mit freundlichen Grüßen
Sebastian Hagedorn

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

rfk

2018-05-14 16:49:59 UTC

Permalink

Hello,

just for information, the update to version 3.0.6 did not fix the
problem for me, I'm still having the same issue. I had to apply my
workaround (create symbolic links) to make it work again.

Regards,
Ralf

Post by rfk
Hi Ken,
my distribution is Gentoo and my previous cyrus version was 2.5.10. Then
I upgraded to 3.0.4 but this ebuild (though marked stable by the Gentoo
devs) was terminating "abnormally" all the time. Finally I upgraded to
3.0.5 which is running fine until now - well, except for sieve.
Regards,
Ralf

Post by rfk
Hey Sebastian,
thanks a lot for your feedback, seems like we're coming closer.
Although I'm not sure what port 24 is supposed to be in your swaks
command I just sent myself an e-mail while strace was attached to
cyrusmaster.
readlink("/var/imap/sieve/u/user^domain^de/defaultbc",
0x7fff7fa31d00, 4096) = -1 ENOENT (No such file or directory)
Then I recognized that the users in /var/imap/sieve/ are actually
separated by a dot ('.') instead of a caret ('^') as opposed to
/var/spool/imap/ where the users are separated by a caret. After
manually renaming the sieve directory of my user to the caret
notation the filtering was finally working.
However Roundcube then tells me that no filters are available. Once I
used Roundcube (or sieveshell) my sieve user directory is existing
twice, once empty in dot notation and once with the filters in caret
notation.
1. lmtpd (?) is using the caret notation when looking up sieve scripts.
2. timsieved (?) is using the dot notation when creating sieve scripts.
Which configuration parameters affect this behaviour?
Regards,
Ralf

Post by Sebastian Hagedorn
Not really. My best suggestion would be to strace the lmtp process
to see if the sieve scripts are even found and opened. On a test
strace -f -ff -o /tmp/lmtptrace -p <PID of cyrus-master>
It might be better to prefork an lmtpd and to specifically strace
that pid, but in my case it didn't matter.
swaks --protocol lmtp -t name-of-a-user-with-sieve -s localhost -p 24
lmtptrace.43344:open("/lib64/libcyrus_sieve.so.0",
O_RDONLY|O_CLOEXEC) = 5
lmtptrace.43344:readlink("/var/lib/imap/sieve/E/xxx/defaultbc",
"smartsieve.bc", 4096) = 13
lmtptrace.43344:open("/var/lib/imap/sieve/E/xxx/smartsieve.script",
O_RDWR) = 12
lmtptrace.43344:stat("/var/lib/imap/sieve/E/xxx/smartsieve.bc",
{st_mode=S_IFREG|0600, st_size=604, ...}) = 0
lmtptrace.43344:stat("/var/lib/imap/sieve/E/xxx/smartsieve.bc",
{st_mode=S_IFREG|0600, st_size=604, ...}) = 0
lmtptrace.43344:open("/var/lib/imap/sieve/E/xxx/smartsieve.bc",
O_RDONLY) = 13
lmtptrace.43344:stat("/var/lib/imap/sieve/E/xxx/defaultbc",
{st_mode=S_IFREG|0600, st_size=604, ...}) = 0
lmtptrace.43344:open("/var/lib/imap/sieve/E/xxx/defaultbc",
O_RDONLY) = 12
Maybe try that and report back what happens on your system.

Post by rfk
Hello,
still no other ideas?
Ralf

Post by rfk
Hi,
I'm so sorry, don't know how this could happen, but the last line of my
altnamespace: 0
So it is already disabled.
Ralf

Post by Michael Menge
Hi,

Post by rfk
Hi,
# grep -v ^# roundcube.script
require ["imapflags","fileinto"];
if anyof (header :contains "Subject" "test")
{
        setflag "\\Seen";
        fileinto "INBOX/Trash";
        stop;
}
# sievec roundcube.script test
#
# grep -vE "^#|^$" /etc/imapd.conf
admins: cyrus
configdirectory: /var/imap
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
duplicate_db_path: /run/cyrus/deliver.db
ptscache_db_path: /run/cyrus/ptscache.db
statuscache_db_path: /run/cyrus/statuscache.db
tls_sessions_db_path: /run/cyrus/tls_sessions.db
defaultpartition: default
partition-default: /var/spool/imap
sievedir: /var/imap/sieve
lmtpsocket: /var/imap/socket/lmtp
idlesocket: /var/imap/socket/idle
notifysocket: /var/imap/socket/notify
syslog_prefix: cyrus
httpmodules: caldav carddav
hashimapspool: true
virtdomains: no
defaultdomain: localhost
popminpoll: 1
allowplaintext: yes
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
sasl_auto_transition: no
sieve_extensions: fileinto reject vacation imapflags notify include
envelope body relational regex subaddress copy
tls_session_timeout: 1440
tls_client_ca_file:     /.../ca_file.pem
tls_server_cert:        /.../cert.pem
tls_server_key:         /.../key.pem
Any other ideas?

the altnamespace default cahnged in 3.0. so you should disable it in
imapd.conf or change the fileinto to "Trash"
Quoting imapd.conf manpage
====================
altnamespace: 1
   Use the alternate IMAP namespace, where personal folders reside at
the same level in the hierarchy as INBOX.
   This option ONLY applies where interaction takes place with the
client/user. Currently this is limited to
   the IMAP protocol (imapd) and Sieve scripts (lmtpd). This option
does NOT apply to admin tools such as
   cyradm (admins ONLY), reconstruct, quota, etc., NOR does it affect
LMTP delivery of messages directly to
   mailboxes via plus-addressing. The default changed in 3.0 from off
to on.
===================

Post by rfk
Ralf

Post by rfk
Hello,
recently I updated my cyrus-imapd installation from verion 2.5.10 to
version 3.0.5. Unfortunately the sieve filters stopped working since
then. The rest is working fine.
The filters are listed in the sieveshell and all incoming mails get
tagged with "X-Sieve: CMU Sieve 3.0". However even the simplest rules
are just getting ignored. In the logs I cannot find any related entries.
My configuration settings regarding "unixhierarchysep" (on) and
"altnamespace" (off) have not changed since version 2.5.10 (I configured
them accordingly in version 3.0.5). I followed the entire upgrade guide
by the way.
Furthermore I also tried adding a completely new account with a very
simple new sieve script (mark as seen and move to folder; created by
Roundcube) but nothing happens to incoming mails.
Is there anything else I can check? Is it possible to raise the log
level for sieve?
Thanks in advance.
Ralf
----
Cyrus Home Page: http://www.cyrusimap.org/
http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-----------------------------------------------------------------------
---------
M.Menge
Tel.: (49) 7071/29-70316 Universität
Tübingen Fax.: (49) 7071/29-5912
Wächterstraße 76
72074 Tübingen
----
Cyrus Home Page: http://www.cyrusimap.org/
http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Mit freundlichen Grüßen
Sebastian Hagedorn

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
http