···706706 year = {2020},
707707}
708708709709+@inbook{lodi-2013-variability,
710710+ author = {Andrea Lodi and Andrea Tramontani},
711711+ title = {Performance Variability in Mixed-Integer Programming},
712712+ booktitle = {Theory Driven by Influential Applications},
713713+ chapter = {Chapter 1},
714714+ year = 2013,
715715+ pages = {1-12},
716716+ doi = {10.1287/educ.2013.0112},
717717+ URL = {https://pubsonline.informs.org/doi/abs/10.1287/educ.2013.0112},
718718+ eprint = {https://pubsonline.informs.org/doi/pdf/10.1287/educ.2013.0112},
719719+}
720720+709721@article{lougee-heimer-2003-coin,
710722 author = {Robin Lougee{-}Heimer},
711723 title = {The Common Optimization INterface for Operations Research: Promoting
+4
assets/listing/half_alldiff.mzn
···11+predicate all_different(array[int] of var int: x) =
22+ forall(i,j in index_set(x) where i < j)(
33+ x[i] != x[j]
44+ );
+5
assets/listing/half_impli.mzn
···11+predicate impli(
22+ var bool: x ::promise_ctx_antitone,
33+ var bool: y ::promise_ctx_monotone
44+) =
55+ not x \/ y;
+15
assets/listing/half_reif_check.mzn
···11+predicate _int_lin_le_imp(
22+ array[int] of int: c,
33+ array[int] of var int: x,
44+ int: d,
55+ var bool: b
66+) =
77+ if is_fixed(b) then
88+ if fix(b) then
99+ int_lin_le(c, x, d)
1010+ else
1111+ true
1212+ endif
1313+ else
1414+ int_lin_le_imp(c, x, d, b)
1515+ endif;
+2-2
chapters/2_background.tex
···155155 \item and the \gls{objective} is included as a solving goal.
156156\end{itemize}
157157158158-More complex models also include definitions for custom types, user defined functions, and a custom output format.
158158+More complex models also include definitions for custom types, user-defined functions, and a custom output format.
159159These items are not constrained to occur in any particular order.
160160We briefly discuss the most important model items.
161161Note that these items already refer to \minizinc{} expressions, which will be discussed in \cref{subsec:back-mzn-expr}.
···180180The type of a \parameter{} can similarly be marked by the \mzninline{par} keyword, but this is not required.
181181These types are used both as normal \parameters{} and as \variables{}.
182182To better structure models, \minizinc{} allows collections of these types to be contained in \glspl{array}.
183183-Unlike other languages, \glspl{array} have a user defined index set, which can start at any value, but they have to be a continuous range.
183183+Unlike other languages, \glspl{array} have a user-defined index set, which can start at any value, but they have to be a continuous range.
184184For example, the following declaration declares an array going from 5 to 10 of new Boolean \variables{}.
185185186186\begin{mzn}
+53-69
chapters/4_half_reif.tex
···30303131\begin{enumerate}
3232 \item \Gls{rewriting} using \glspl{half-reif} naturally produces \glspl{rel-sem}.
3333- \item \Glspl{propagator} for a \glspl{half-reif} can often be constructed by merely altering a \gls{propagator} implementation for its regular \constraint{}.
3333+ \item \Glspl{propagator} for a \glspl{half-reif} can usually be constructed by merely altering a \gls{propagator} implementation for its regular \constraint{}.
3434 \item \Gls{half-reif} does not often interact with its \gls{cvar}, limiting the amount of triggered \glspl{propagator} that are known to be unable to prune any \domains{}.
3535\end{enumerate}
3636···7676\end{description}
77777878As previously explained, \gls{half-reif} can be used for expressions in \posc{} context.
7979-Although expressions in a \negc{} context cannot be directly \gls{half-reified}, the negation of a expression in a \negc{} context can be \gls{half-reified}.
7979+Although expressions in a \negc{} context cannot be directly \gls{half-reified}, the negation of an expression in a \negc{} context can be \gls{half-reified}.
80808181\begin{example}
8282 Consider, for example, the following \constraint{}.
···9898\end{example}
9999100100Expressions in a \mixc{} context are in a position where \gls{half-reif} is impossible.
101101-Only full \gls{reif} can be used for expressions in that are in this context.
101101+Only full \gls{reif} can be used for expressions that are in this context.
102102This occurs, for example, when using an exclusive or expression in a \constraint{}.
103103The value that one side must take directly depends on the value that the other side takes.
104104Each side can thus be forced to be true or false.
···122122123123\begin{algorithm}
124124125125- \KwIn{A function \(check\), that returns false when the \constraint{} \(c\) cannot be satisfied, a function \(prune\), that eliminates values from \variable{} \glspl{domain} that violate\glsadd{violated} the \constraint{} \(c\), and a Boolean control \variable{} \texttt{b}.
125125+ \KwIn{A function \(check\), that returns false when the \constraint{} \(c\) cannot be satisfied, a function \(prune\), that eliminates values from \variable{} \glspl{domain} that violate\glsadd{violated} the \constraint{} \(c\), and a Boolean \gls{cvar} \texttt{b}.
126126 }
127127 \KwResult{This pseudo code propagates the \gls{half-reif} of \(c\) (\ie{} \(\texttt{b} \implies\ c\)).}
128128···191191For all these \glspl{reif}, its replacement by a \gls{half-reif} can remove half of the implications required for the \gls{reif}.
192192193193For \gls{sat} solvers, a \gls{decomp} for a \gls{half-reif} can be created from its regular \gls{decomp}.
194194-Any \constraint{} \(c\) will \gls{decomp} into \gls{cnf}.
194194+Any \constraint{} \(c\) will \gls{decomp} into \gls{cnf} of the following form.
195195\[ c = \forall_{i} \exists_{j} lit_{ij} \]
196196The \gls{half-reif}, with \gls{cvar} \texttt{b}, could take the following encoding.
197197\[ \texttt{b} \implies c = \forall_{i} \neg \texttt{b} \lor \exists_{j} lit_{ij} \]
···340340341341\begin{example}
342342343343- Consider the following user-defined \minizinc{} implementation of a logical implication.
343343+ Consider the user-defined \minizinc{} implementation of a logical implication in \cref{lst:half-impli}.
344344345345- \begin{mzn}
346346- predicate impli(
347347- var bool: x ::promise_ctx_antitone,
348348- var bool: y ::promise_ctx_monotone
349349- ) =
350350- not x \/ y;
351351- \end{mzn}
345345+ \begin{listing}
346346+ \mznfile{assets/listing/half_impli.mzn}
347347+ \caption{\label{lst:half-impli} A user-defined predicate of a logical implication using \glspl{annotation} to define the context usage of its arguments.}
348348+ \end{listing}
352349353353- The annotations placed on the argument of the \mzninline{impli} function will apply the same context transformations as the \mzninline{->} operator shown in \cref{alg:arg-ctx}.
350350+ The \glspl{annotation} placed on the argument of the \mzninline{impli} function will apply the same context transformations as the \mzninline{->} operator shown in \cref{alg:arg-ctx}.
354351 In term of context analysis, this function now is equivalent to the \minizinc{} operator.
355352356353\end{example}
···373370 \Case{\mzninline{ > }, \mzninline{>= }, \mzninline{- }}{
374371 \Return{\tuple{\changepos{}ctx, \changeneg{}ctx}}
375372 }
376376- \Case{\mzninline{ <-> }, \mzninline{xor }, \mzninline{* }}{
373373+ \Case{\mzninline{ <-> }, \mzninline{= }, \mzninline{xor }, \mzninline{* }}{
377374 \Return{\tuple{\mixc, \mixc}}
378375 }
379376 \Case{\mzninline{ /\ }}{
···409406 Note that this can only occur when a \gls{native} \constraint{} does not define a \gls{reif}.
410407\end{enumerate}
411408412412-The (Access) and (ITE) rules show the context inference for \gls{array} access and if-then-else expressions respectively.
409409+The (Access) and (ITE) rules show the context inference for \gls{array} access and \gls{conditional} expressions respectively.
413410Their inner expressions are evaluated in \(\changepos{}ctx\).
414411The inner expressions cannot be simply be evaluated in \(ctx\), because it is not yet certain which expression will be chosen.
415412This is important for when \(ctx\) is \rootc{}, since we, at compile time, cannot say which expression will hold globally.
···505502\label{subsec:half-?root}
506503507504In the previous section, we briefly discussed the context transformations for the (Access) and (ITE) rules in \cref{fig:half-analysis-expr}.
508508-Different from the rules described, when an \gls{array} access or if-then-else expression is found in \rootc{} context, it often makes sense to evaluate its sub-expression in \rootc{} context as well.
505505+Different from the rules described, when an \gls{array} access or \gls{conditional} expression is found in \rootc{} context, it often makes sense to evaluate its sub-expression in \rootc{} context as well.
509506It is, however, not always safe to do so.
510507511508\begin{example}
···514511 For example, consider the following \microzinc{} fragment.
515512516513 \begin{mzn}
517517- \constraint{} if b then
514514+ constraint if b then
518515 F(x, y, z)
519516 else
520517 G(x, y, z)
521518 endif;
522519 \end{mzn}
523520524524- In this case, since only one side of the if-then-else expression is evaluated, the compiler can output calls to the \rootc{} variant of the functions.
521521+ Let us assume that \mzninline{b} is a \parameter{}, but that its value is not known during the compilation from \minizinc{} to \microzinc{}.
522522+ In this case, we know that only one side of the \gls{conditional} expression will evaluated, and that call will then globally constrain the \cmodel{}.
523523+ As such, the \compiler{} can output calls to the \rootc{} variant of the functions.
525524 This will enforce the \constraint{} in the most efficient way.
526525527526 Things, however, change when the situation gets more complex.
···536535 } in ret;
537536 \end{mzn}
538537539539- One side of the if-then-else expression is also used in a disjunction.
538538+ One side of the \gls{conditional} expression is also used in a disjunction.
540539 If \mzninline{b} evaluates to \mzninline{true}, then \mzninline{p} is evaluated in \rootc{} context, and \mzninline{p} can take the value \mzninline{true} in the disjunction.
541540 Otherwise, \mzninline{q} is evaluated in \rootc{} context, and \mzninline{p} in the disjunction must be evaluated in \posc{} context.
542542- It is, therefore, not safe to assume that all sides of the if-then-else expressions are evaluated in \rootc{} context.
541541+ In this situation, it is not safe for the \compiler{} to output calls for the \rootc{} variants of these calls.
543542\end{example}
544543545544Using the \changepos{} transformation for sub-expression contexts is safe, but it places a large burden on the \solver{}.
546545The solver performs better when the no \gls{reif} has to be used.
547546548548-To detect situation where the sub-expression are only used in an \gls{array} access or if-then-else expression we introduce the \mayberootc{} context.
547547+To detect situation where the sub-expression are only used in an \gls{array} access or \gls{conditional} expression we introduce the \mayberootc{} context.
549548This context functions as a ``weak'' \rootc{} context.
550549If it is joined with any other context, then it acts as \posc{}.
551550The extended join operation is shown in \cref{fig:half-maybe-join}.
···580579 \caption{\label{fig:half-analysis-maybe-root} Updated context inference rules for \mayberootc{}.}
581580\end{figure*}
582581583583-\Cref{fig:half-analysis-maybe-root} shows the additional inference rules for \gls{array} access and if-then-else expressions.
582582+\Cref{fig:half-analysis-maybe-root} shows the additional inference rules for \gls{array} access and \gls{conditional} expressions.
584583Looking back at \cref{ex:half-maybe-root}, these additional rules and updated join operation will ensure that the first case will correctly use \rootc{} context calls.
585584For the second case, however, it detects that \mzninline{p} is used in both \posc{} and \mayberootc{} context.
586585Therefore, it will output the \posc{} call for the right hand side of \mzninline{p}, even when \mzninline{b} evaluates to \mzninline{true}.
···590589\section{Rewriting and Half Reification}%
591590\label{sec:half-rewriting}
592591593593-During the \gls{rewriting} process the contexts assigned to the different expressions can be used directly to determine if and how a expression has to be \gls{reified}.
592592+During the \gls{rewriting} process the contexts assigned to the different expressions can be used directly to determine if and how an expression has to be \gls{reified}.
594593595594\begin{example}
596595\label{ex:half-rewriting}
···626625 constraint bool_clause([b1, b2], []); % b1 \/ b2
627626 constraint bool_clause_imp([b3], [y], b1); % b1 -> (y -> b3)
628627 constraint f_imp(x, b3); % b3 -> f(x)
629629- constraint int_ne_imp(x, 5, b4); % b4 -> x != 5
628628+ constraint int_ne_imp(x, 5, b2); % b2 -> x != 5
630629 \end{mzn}
631630632631 We are able to replace the two \glspl{reif} and push the negation inwards, transforming the equals \constraint{} into a not equals \constraint{}.
···634633 If the Boolean \mzninline{y} was not further \constraint{} in the problem, then we could further reduce it to the following \constraints{}.
635634636635 \begin{mzn}
637637- constraint bool_clause([y, b2], []); % y \/ b2
638638- constraint f_imp(x, y); % b3 -> f(x)
639639- constraint int_ne_imp(x, 5, b4); % b4 -> x != 5
636636+ constraint bool_clause([y, b2], []); % y \/ b2
637637+ constraint f_imp(x, y); % y -> f(x)
638638+ constraint int_ne_imp(x, 5, b2); % b2 -> x != 5
640639 \end{mzn}
641640642641\end{example}
···650649The \gls{rewriting} with \gls{half-reif} also interacts with some of the optimisation methods used during the \gls{rewriting} process.
651650Most importantly, \gls{half-reif} has to be considered when using \gls{cse} and \gls{propagation} can change the context of expression.
652651In \cref{subsec:half-cse} we will discuss how \gls{cse} can be adjusted to handle \gls{half-reif}.
653653-Finally, in \cref{subsec:half-dyn-context} we will discuss how the context in which a expression is executed can be adjusted during the \gls{rewriting} process.
652652+Finally, in \cref{subsec:half-dyn-context} we will discuss how the context in which an expression is executed can be adjusted during the \gls{rewriting} process.
654653655654\subsection{Chain compression}%
656655\label{subsec:half-compress}
···660659The case shown in the example can be generalised to
661660662661\begin{mzn}
663663- b1 -> b2 /\ forall(i in N)(b2 -> c[i])
662662+ constraint b1 -> b2 /\ forall(i in N)(b2 -> c[i])
664663\end{mzn}
665664666665\noindent{}which, if \texttt{b2} has no other usage in the instance, can be resolved to
667666668667\begin{mzn}
669669- forall(i in N)(b1 -> c[i])
668668+ constraint forall(i in N)(b1 -> c[i])
670669\end{mzn}
671670672671\noindent{}after which \texttt{b2} can be removed from the model.
···708707These can be split up into multiple implications.
709708710709\begin{mzn}
711711- b -> forall(x in N)(x)
710710+ constraint b -> forall(x in N)(x)
712711\end{mzn}
713712714713The expression above is logically equivalent to the following expression.
715714716715\begin{mzn}
717717- forall(x in N)(b -> x)
716716+ constraint forall(x in N)(b -> x)
718717\end{mzn}
719718720719Adopting this transformation both simplifies a complicated \constraint{} and possibly allows for the further compression of \glspl{implication-chain}.
721720It should however be noted that although this transformation can increase the number of \constraints{} in the \gls{slv-mod}, it generally increases the \gls{propagation} efficiency.
722721723723-To adjust the algorithm to simplify implied conjunctions more introspection from the \minizinc{} \compiler{} is required.
722722+To adjust the algorithm to simplify implied conjunctions, more introspection from the \minizinc{} \compiler{} is required.
724723The \compiler{} must be able to tell if a \variable{} is (only) a \gls{cvar} of a reified conjunction and what the elements of that conjunction are.
725725-In the case where a \variable{} has one incoming edge, but it is marked as used in other \constraint{}, we can now check if it is only a \gls{cvar} for a \gls{reified} conjunction and perform the transformation in this case.
724724+In the case where a \variable{} has one incoming edge, but it is marked as used in another \constraint{}, we can now check if it is only a \gls{cvar} for a \gls{reified} conjunction and perform the transformation in this case.
726725727726\subsection{Common Sub-expression Elimination}%
728727\label{subsec:half-cse}
···759758The dependency tracking through the use of \constraints{} attached to \variables{} ensures no defining \constraints are left in the model.
760759This ensures that all \variables{} and \constraints{} created the earlier version are correctly removed.
761760762762-Because the expression itself is changed when a negation is moved inwards, it may not always be clear when the same expression is used in both \posc{} and negc{} context.
761761+Because the expression itself is changed when a negation is moved inwards, it may not always be clear when the same expression is used in both \posc{} and \negc{} context.
763762This problem is solved by introducing a canonical form for expressions where negations can be pushed inwards.
764763In this form the result of \gls{rewriting} an expression and its negation are collected in the same place within the \gls{cse} table.
765764If it is found that for an expression that is about to be \gls{half-reified} there already exists an \gls{half-reif} for its negation, then we instead evaluate the expression in \mixc{} context.
···771770 When its expression is negated, pushing the negation inwards will result in a \mzninline{!=} operator, a \mzninline{int_ne} call.
772771 The opposite happens when a negation is pushed inwards for an expression using \mzninline{!=} operator.
773772 So to ensure that a \negc{} occurrence of \mzninline{int_eq} and a \posc{} occurrence of \mzninline{int_ne} use the same \gls{cvar} they are both mapped to \mzninline{int_eq} in the \gls{cse} table.
774774- The mapping ensures that the context is correctly transformed when accessing the \gls{cse} table for a \mzninline{int_ne} call.
773773+ The mapping ensures that the context is correctly transformed when accessing the \gls{cse} table for an \mzninline{int_ne} call.
775774776775\end{example}
777776···799798 constraint b -> (2*x = y);
800799 \end{mzn}
801800802802- Since the \domain{} of \mzninline{x} is strictly smaller than the \domain{} of \mzninline{y}, \gls{propagation} of \mzninline{b} will set it to the value \mzninline{true}.
801801+ Since the values in the \domain{} of \mzninline{x} are strictly smaller than the values in the \domain{} of \mzninline{y}, \gls{propagation} of \mzninline{b} will set it to the value \mzninline{true}.
803802 This however means that the \constraint{} is equivalent to the following \constraint{}.
804803805804 \begin{mzn}
···811810\end{example}
812811813812The situation shown in the example is the most common change of context.
814814-If the control \variable{} of a \gls{reif} is fixed, the context often changes to either \rootc{} or a negated \rootc{} context.
813813+If the \gls{cvar} of a \gls{reif} is fixed, the context often changes to either \rootc{} or a negated \rootc{} context.
815814If, on the other hand, the \gls{cvar} of a \gls{half-reif} is fixed, then either the context becomes \rootc{} or the \constraint{} already holds.
816815Since direct \constraints{} are strongly preferred over any form of \gls{reif}, it is important to dynamically pick the correct form during the \gls{rewriting} process.
817816818817This problem can be solved by the \compiler{}.
819818For each \gls{reif} and \gls{half-reif} the \compiler{} introduces another layer of \gls{decomp}.
820819In this layer, it checks its \gls{cvar}.
821821-If the control \variable{} is already fixed, then it rewrites itself into its form in another context.
820820+If the \gls{cvar} is already fixed, then it rewrites itself into its form in another context.
822821Otherwise, it behaves as it would have done normally.
823822The \gls{cvar} is thus used to communicate the change in context.
824823···826825827826 Let's assume the \compiler{} finds a call to \mzninline{int_lin_le} in \posc{} context.
828827 Instead of outputting the call to \mzninline{int_lin_le_imp} directly, it will instead output a call to \mzninline{_int_lin_le_imp}.
829829- This predicate is then generated as follows:
828828+ This predicate is then generated as shown in \cref{lst:half-check-reif}.
829829+ This new predicate (calls) can then be rewritten normally.
830830831831- \begin{mzn}
832832- predicate _int_lin_le_imp(
833833- array[int] of int: c,
834834- array[int] of var int: x,
835835- int: d,
836836- var bool: b
837837- ) =
838838- if is_fixed(b) then
839839- if fix(b) then
840840- int_lin_le(c, x, d)
841841- else
842842- true
843843- endif
844844- else
845845- int_lin_le_reif(c, x, d, b)
846846- endif;
847847- \end{mzn}
831831+ \begin{listing}
832832+ \mznfile{assets/listing/half_reif_check.mzn}
833833+ \caption{\label{lst:half-check-reif}A generated predicate for \mzninline{int_lin_le_imp} that checks its \gls{cvar} to ensure a \gls{half-reif} is still required.}
834834+ \end{listing}
848835849849- This new predicate can then be compiled using the normal methods.
850836\end{example}
851837852838\section{Experiments}
···889875Since \gls{chuffed} is a \gls{lcg} \solver{}, the explanations created by the \gls{propagator} have to be adjusted as well.
890876These adjustments happen in a similar fashion to the adjustments of the general algorithm: explanations used for the violation of the \constraint{} can now be used to set the \gls{cvar} to \mzninline{false} and the explanations given to prune a variable are appended by requirement that the \gls{cvar} is \mzninline{true}.
891877892892-In our second configuration the \mzninline{all_different} \constraint{} is enforced using the following \gls{decomp}.
893893-894894-\begin{mzn}
895895- predicate all_different(array[int] of var int: x) =
896896- forall(i,j in index_set(x) where i < j)(
897897- x[i] != x[j]
898898- );
899899-\end{mzn}
900900-878878+In our second configuration the \mzninline{all_different} \constraint{} is enforced using the \gls{decomp} shown in \cref{lst:half-alldiff}.
901879The \mzninline{!=} \constraints{} produced by this redefinition are \gls{reified}.
902880Their conjunction, then represent the \gls{reif} of the \mzninline{all_different} \constraint{}.
881881+882882+\begin{listing}
883883+ \mznfile{assets/listing/half_alldiff.mzn}
884884+ \caption{\label{lst:half-alldiff}The standard \gls{decomp} for \mzninline{all_different} in the \minizinc{} standard library.}
885885+\end{listing}
886886+903887904888\begin{table}
905889 \begin{center}
···10391023However, we can imagine that the removed \constraints{} in some cases help the \gls{mip} solver.
10401024An important technique used by \gls{mip} solvers is to detect certain pattern, such as cliques, during the pre-processing of the \gls{mip} instance.
10411025Some patterns can only be detected when using a full \gls{reif}.
10421042-Furthermore, the performance of \gls{mip} solvers is often dependent on the order and form in which the \constraints{} are given \autocite{fischetti-2014-erratiscism}.
10261026+Furthermore, the performance of \gls{mip} solvers is often dependent on the order and form in which the \constraints{} are given \autocite{lodi-2013-variability,fischetti-2014-erratiscism}.
10431027With the usage of the \gls{aggregation} and \gls{del-rew}, these can be exceedingly different when using \gls{half-reif}.
1044102810451029The solving statistics for \gls{openwbo} might be most positive.
+3-3
chapters/4_half_reif_preamble.tex
···11\noindent{}Whether a \constraint{} has to be \gls{reified} is a crucial decision made during \gls{rewriting}.
22\minizinc{} allows us to write complex \constraints{} that in the process of \gls{rewriting} result in multiple \constraints{} that reason about each other.
33-However, determining whether a \constraint{} holds, \(\texttt{b} \leftrightarrow{} c\), requires significantly more effort from the \solver{} than merely enforcing the \constraint{} \(c\).
33+However, determining whether a \constraint{} holds, \(b \leftrightarrow{} c\), requires significantly more effort from the \solver{} than merely enforcing the \constraint{} \(c\).
44A \gls{reified} \constraint{} results in a larger \gls{decomp} or a more complex \gls{native} \constraint{}.
55It it thus important that \gls{rewriting} avoids the use of \gls{reif}, whenever it is not required.
6677In this chapter we present an extended \gls{reif} analysis to help minimise the required \solver{} effort.
88Not only does it consider if a \constraint{} must be \gls{reified} or not, but it also considers whether it could instead be \gls{half-reified}.
991010-The notion of \gls{half-reif} \(\texttt{b} \rightarrow{} c\) for a \constraint{} \(c\) was introduced by \textcite{feydy-2011-half-reif}.
1111-It is shown \gls{half-reif} can relief some of the problems and expenses of the use of \gls{reif}.
1010+The notion of \gls{half-reif} \(b \rightarrow{} c\) for a \constraint{} \(c\) was introduced by \textcite{feydy-2011-half-reif}.
1111+It is shown \gls{half-reif} can relieve some of the problems and expenses of the use of \gls{reif}.
1212It is not always possible, however, to use a \gls{half-reif} instead of a \gls{reif}.
1313The authors identify the conditions required for its usage and provide an algorithm that rewrites a subset of the \minizinc{} language using the technique.
1414Crucially, because of the omission of \gls{let} and multiple occurrences of identifiers, this algorithm does not directly generalise to the full \minizinc{} language.